[Wednesday 01 September 2010] [00:12:01] Quit poswald has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [00:48:28] Quit xrfang has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [01:08:16] Quit
xrfang_ has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:36:48] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 01 September 2010] [01:39:43] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:54:38] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:58:11] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47c064f 10/ (src/fq.cpp src/lb.cpp): hangup when closing socket with no pipes attached – fixed -
http://bit.ly/9iy2zX[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * raaa0761 10/ src/pair.cpp : pipe being attached to the PAIR socket during its termination process is immediately asked to terminate itself -
http://bit.ly/bwIpo1[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * rdb73c76 10/ src/pub.cpp : assert when pipe attaches to PUB socket in process of termination – fixed -
http://bit.ly/bCxg7v[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * rce0972d 10/ (src/ctx.cpp src/ctx.hpp src/object.cpp src/object.hpp): context creates an inproc endpoint (‘inproc://log’) to distribute 0MQ’s log messages - http://bit.ly/9YrEbe
[Wednesday 01 September 2010] [02:25:20] Quit sustrik has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [02:29:02] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [02:30:13] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [03:02:01] <guido_g> howdy
[Wednesday 01 September 2010] [03:05:49] <guido_g> cool, logging...
[Wednesday 01 September 2010] [03:09:41] <guido_g> does the core itself use the logging?
[Wednesday 01 September 2010] [03:10:06] * guido_g looks at the patch
[Wednesday 01 September 2010] [03:20:18] <guido_g> hmmm
[Wednesday 01 September 2010] [03:20:23] <guido_g> doesn’t seem so
[Wednesday 01 September 2010] [03:26:11] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [03:27:08] <sustrik> only the core can use the logging
[Wednesday 01 September 2010] [03:27:27] <sustrik> however, atm it’s not logging anything
[Wednesday 01 September 2010] [03:27:46] <sustrik> logs are to be added gradually
[Wednesday 01 September 2010] [03:31:32] <guido_g> well... better then nothing :)
[Wednesday 01 September 2010] [04:50:11] <Mikael_H_Kjaer> Sustrik: I don’t have a small example that triggers the bug I’ve posted about on the mailing list.
[Wednesday 01 September 2010] [04:51:04] <sustrik> hi mikael
[Wednesday 01 September 2010] [04:51:10] <sustrik> that’s a problem
[Wednesday 01 September 2010] [04:51:30] <sustrik> from inspecting the backtrace it looks like it’s not a 0mq problem
[Wednesday 01 September 2010] [04:51:37] <Mikael_H_Kjaer> I know. I have been trying to boil it down
[Wednesday 01 September 2010] [04:51:49] <sustrik> maybe a memory overwrite?
[Wednesday 01 September 2010] [04:52:05] <sustrik> some random code overwriting the socket?
[Wednesday 01 September 2010] [04:52:07] <Mikael_H_Kjaer> Maybe. Currently I am not hitting the issue anymore
[Wednesday 01 September 2010] [04:52:55] <Mikael_H_Kjaer> But I shouldn’t have any wild pointers or memory accesses in my code. I am currently using smart pointers for everything and my cycle checker seems to think I am okay too.
[Wednesday 01 September 2010] [04:54:07] <Mikael_H_Kjaer> I haven’t valgrinded my code in a few weeks but I will soon try it with zmq added in.
[Wednesday 01 September 2010] [04:54:47] <sustrik> ok. good
[Wednesday 01 September 2010] [04:55:24] <sustrik> btw, maybe try to compile with debug into so that backtrace is more informative
[Wednesday 01 September 2010] [04:56:37] <Mikael_H_Kjaer> I do :) what additional information do you want
[Wednesday 01 September 2010] [04:58:46] <sustrik> hm
[Wednesday 01 September 2010] [04:59:14] <sustrik> can you possibly add a printf statement to socket’s destructor
[Wednesday 01 September 2010] [04:59:36] <sustrik> so that we know whether the socket was destructed or overwritten?
[Wednesday 01 September 2010] [04:59:58] <sustrik> printing out this pointer would be good so that we can correlate it with the backtrace
[Wednesday 01 September 2010] [05:00:05] <Mikael_H_Kjaer> Not a problem
[Wednesday 01 September 2010] [05:00:22] <Mikael_H_Kjaer> I will get back with more. Any other printf’s you want?
[Wednesday 01 September 2010] [05:01:53] <sustrik> not yet
[Wednesday 01 September 2010] [05:02:02] <sustrik> let’s first see whrther the socket is destructed
[Wednesday 01 September 2010] [05:06:57] Quit
miguelito_ has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [05:12:38] <Mikael_H_Kjaer> sustrik: I can’t get the error to show up any more. I believe my fix of waiting for all sends to finish before attempting a shutdown of zmq has resolved it.
[Wednesday 01 September 2010] [05:12:58] <sustrik> hm
[Wednesday 01 September 2010] [05:13:29] <sustrik> ok, so let’s leave it for now
[Wednesday 01 September 2010] [05:13:40] <sustrik> if you ever see it again, we’ll continue
[Wednesday 01 September 2010] [05:16:20] <Mikael_H_Kjaer> Agreed, I was thinking the same.
[Wednesday 01 September 2010] [05:23:43] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 01 September 2010] [05:28:05] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 01 September 2010] [05:41:42] Quit Mikael_H_Kjaer has left this server (Quit: Page closed).
[Wednesday 01 September 2010] [05:52:02] <mcxx> however, in this setup, the broker is the single point of failure
[Wednesday 01 September 2010] [05:52:54] <mcxx> can the architecture be altered somehowe? e.g. can the clients connect to multiple brokers so that when one goes down, they switch to a working one?
[Wednesday 01 September 2010] [05:54:50] <sustrik> yes, you can load balance via multiple brokers
[Wednesday 01 September 2010] [05:55:08] <sustrik> just start 2 brokers
[Wednesday 01 September 2010] [05:55:17] <sustrik> and connect all components to both of them
[Wednesday 01 September 2010] [05:55:35] <sustrik> you also have to set HWM on clients
[Wednesday 01 September 2010] [05:55:52] <sustrik> so that messages are not queued for the unavailable broker indefinitely
[Wednesday 01 September 2010] [05:56:46] Quit guido_g has left this server (Client Quit).
[Wednesday 01 September 2010] [05:57:14] <mcxx> what’s HWM? Google doesn’t show reasonable results
[Wednesday 01 September 2010] [05:57:44] <mcxx> oh, I see noe
[Wednesday 01 September 2010] [05:57:45] <mcxx> now
[Wednesday 01 September 2010] [05:59:24] Quit guido_g has left this server (Disconnected by services).
[Wednesday 01 September 2010] [05:59:30] Nick OldNugget is now known as guido_g.
[Wednesday 01 September 2010] [06:39:04] <mcxx> when I set HWM on the client and one of the brokers goes down, will the clients’ sockets find out the broker is no longer available and automatically use others?
[Wednesday 01 September 2010] [06:39:45] <guido_g> huh? what broker?
[Wednesday 01 September 2010] [06:40:12] <guido_g> you mean the pipes of the underlying connections?
[Wednesday 01 September 2010] [06:40:32] <mcxx> guido_g: it’s a follow up to my previous question
[Wednesday 01 September 2010] [06:40:38] <guido_g> as far as i know, full pipes (hwm reached) are skipped
[Wednesday 01 September 2010] [06:41:08] <guido_g> i see
[Wednesday 01 September 2010] [06:41:38] <guido_g> still, full pipes are skipped
[Wednesday 01 September 2010] [06:43:58] <mcxx> re-reading that helps :)
[Wednesday 01 September 2010] [06:46:03] <guido_g> :)
[Wednesday 01 September 2010] [07:13:31] Quit rbraley has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [07:26:08] Quit sustrik has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [07:32:01] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6a0c323 10/ (builds/msvc/libzmq/libzmq.vcproj src/ctx.cpp): MSVC build fixed -
http://bit.ly/cEMTGs[Wednesday 01 September 2010] [07:39:18] <pieterh> I’ve posted a blog that hopefully clears up the naming confusion for once and for all...
[Wednesday 01 September 2010] [07:39:27] <pieterh> anyone from Denmark here?
[Wednesday 01 September 2010] [07:40:01] Quit ivan has left this server (Quit: Coyote finally caught me).
[Wednesday 01 September 2010] [07:43:28] <guido_g> :)
[Wednesday 01 September 2010] [08:04:50] <pieterh> sustrik: random question about 2.1
[Wednesday 01 September 2010] [08:05:21] <sustrik> yes?
[Wednesday 01 September 2010] [08:07:08] <pieterh> how solid is it?
[Wednesday 01 September 2010] [08:07:18] <pieterh> should I start to be using it for the examples in the guide?
[Wednesday 01 September 2010] [08:08:01] <sustrik> it haven’t been tested extensively
[Wednesday 01 September 2010] [08:08:06] <sustrik> but the API is the same
[Wednesday 01 September 2010] [08:08:34] <sustrik> so, for the guide, it’s irrelevant which one you use
[Wednesday 01 September 2010] [08:08:46] <pieterh> there’s the ‘insert random sleep here’ which i’d like to remove
[Wednesday 01 September 2010] [08:08:55] <sustrik> ah, yes, that can go with 2.1
[Wednesday 01 September 2010] [08:09:00] <pieterh> i’ll start to test it then
[Wednesday 01 September 2010] [08:09:11] <pieterh> it’d be fun to release a 2.1.0 unstable IMO
[Wednesday 01 September 2010] [08:09:12] <sustrik> btw, there’s a test suite in the package now
[Wednesday 01 September 2010] [08:09:24] <pieterh> especially if you have stuff like sys:// in there...
[Wednesday 01 September 2010] [08:09:24] <sustrik> so if you write a test
[Wednesday 01 September 2010] [08:09:28] <sustrik> jsut put it in there
[Wednesday 01 September 2010] [08:10:05] <sustrik> well, the infrastructure is there, but no notifications are passed atm
[Wednesday 01 September 2010] [08:10:08] <pieterh> :-) i saw some emails on that but missed the original discussion
[Wednesday 01 September 2010] [08:10:15] <pieterh> great idea to have a test suite!
[Wednesday 01 September 2010] [08:10:28] <sustrik> it’s in tests subdir
[Wednesday 01 September 2010] [08:10:34] <sustrik> just put your file there
[Wednesday 01 September 2010] [08:10:39] <sustrik> add it to Makefila.am
[Wednesday 01 September 2010] [08:10:45] <sustrik> and make check will run it
[Wednesday 01 September 2010] [08:10:47] <pieterh> ack
[Wednesday 01 September 2010] [08:11:42] <guido_g> ah, while talking about tests
[Wednesday 01 September 2010] [08:12:10] <guido_g> i plan to write the basic tests for the missing socket types over the weekend
[Wednesday 01 September 2010] [08:12:30] <sustrik> missing socket types?
[Wednesday 01 September 2010] [08:12:40] <sustrik> ah, those not yet in test suite
[Wednesday 01 September 2010] [08:12:47] <guido_g> afaik only req/rep and pair are tested atm
[Wednesday 01 September 2010] [08:12:48] <sustrik> nice
[Wednesday 01 September 2010] [08:13:04] <pieterh>
ZMQ_[Wednesday 01 September 2010] [08:13:05] <guido_g> if there are any corner cases that should be tested, please leave me a note
[Wednesday 01 September 2010] [08:13:14] <pieterh> the secret socket typ
[Wednesday 01 September 2010] [08:13:18] <pieterh>
*type
[Wednesday 01 September 2010] [08:13:27] <pieterh> if you can’t spell it, you can’t use it...
[Wednesday 01 September 2010] [08:13:35] <guido_g> pieterh seems to be in a funny mood today
[Wednesday 01 September 2010] [08:13:36] <guido_g> :)
[Wednesday 01 September 2010] [08:13:56] * pieterh had too many beers with mato last night, his head is not switched on right
[Wednesday 01 September 2010] [08:13:56] <sustrik> you really got the letter or have you made it up?
[Wednesday 01 September 2010] [08:14:05] <pieterh> that’s a real letter
[Wednesday 01 September 2010] [08:14:19] <pieterh> i couldn’t invent something _that_ ... unique
[Wednesday 01 September 2010] [08:14:22] <guido_g> justifies the number of beers you had
[Wednesday 01 September 2010] [08:14:40] <pieterh> that’s why the guide made that strange reference to annoying the Danes
[Wednesday 01 September 2010] [08:15:07] <guido_g> ahhh
[Wednesday 01 September 2010] [08:15:54] <pieterh> next we’re going to get email from the mathematicians accusing us of soiling Bourbaki’s name
[Wednesday 01 September 2010] [08:16:42] <pieterh> guido_g: what language are you using for the tests?
[Wednesday 01 September 2010] [08:16:52] <guido_g> c++
[Wednesday 01 September 2010] [08:17:09] <guido_g> but they can be in any language the core understands
[Wednesday 01 September 2010] [08:17:11] <pieterh> there’s one example that would work for push/pull and pair
[Wednesday 01 September 2010] [08:17:35] <guido_g> for these sockets we have tests already
[Wednesday 01 September 2010] [08:17:38] <pieterh> the mtrelay
[Wednesday 01 September 2010] [08:17:58] <pieterh> you just said only req/rep and pair...?
[Wednesday 01 September 2010] [08:18:21] <guido_g> oh sorry
[Wednesday 01 September 2010] [08:18:33] <guido_g> parsed the pub/sub wrong
[Wednesday 01 September 2010] [08:18:46] <guido_g> push/pull
[Wednesday 01 September 2010] [08:18:48] <pieterh> agh, I realized that example in fact doesn’t use push/pull finally, i switched to using pair
[Wednesday 01 September 2010] [08:18:51] * guido_g needs new glases
[Wednesday 01 September 2010] [08:18:58] * pieterh needs a new head
[Wednesday 01 September 2010] [08:19:26] <guido_g> and i thought they do have good beer in .sk
[Wednesday 01 September 2010] [08:19:43] <pieterh> it’s ok, not real beer like we make in Belgium
[Wednesday 01 September 2010] [08:19:54] <guido_g> ah
[Wednesday 01 September 2010] [08:20:02] <pieterh> kind of halfway between real beer (belgian) and fake beer (anglo)
[Wednesday 01 September 2010] [08:20:08] <pieterh> just kidding
[Wednesday 01 September 2010] [08:20:19] <pieterh> it’s the home of beer, of course
[Wednesday 01 September 2010] [08:20:57] <pieterh> ZeroMQ is a big hit in Japan it seems
[Wednesday 01 September 2010] [08:21:15] <pieterh> RT !heavenshell: perl ZeroMQ
[Wednesday 01 September 2010] [08:21:27] <guido_g> ahh... sure
[Wednesday 01 September 2010] [08:21:42] <sustrik> there are some japanese guys working on the perl binding afaiu
[Wednesday 01 September 2010] [08:21:43] <pieterh> I think they’re complaining about the spelling... :-/
[Wednesday 01 September 2010] [08:21:51] <pieterh> my Japanese isn’t so good anymore
[Wednesday 01 September 2010] [08:22:01] Quit jond has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [08:22:13] <sustrik> btw “where 0mq goes, google cannot follow” is a nice slogan
[Wednesday 01 September 2010] [08:22:16] * pieterh wonders about MQ...
[Wednesday 01 September 2010] [08:22:28] <sustrik> wrt distributed vs. centralised services
[Wednesday 01 September 2010] [08:22:38] <sustrik> privacy etc.
[Wednesday 01 September 2010] [08:22:55] <sustrik> MQ would be read NoMQ :)
[Wednesday 01 September 2010] [08:25:48] <pieterh> Hehe, “can’t follow this”...
[Wednesday 01 September 2010] [08:27:34] <pieterh> we need to prepare an ultrasecret joke for next April 1st
[Wednesday 01 September 2010] [08:27:40] <pieterh> where we rename 0MQ to MQ
[Wednesday 01 September 2010] [08:27:54] <pieterh> get lots of people to blog and twitter about it
[Wednesday 01 September 2010] [08:27:59] <pieterh> confuse the heck out of everyone...
[Wednesday 01 September 2010] [08:31:16] <pieterh> sustrik: could you send me a .zip of the zeromq2 master (clean checkout)?
[Wednesday 01 September 2010] [08:31:29] <pieterh> i’m on a slow 2G connection and have failed 4 times to do a git pull
[Wednesday 01 September 2010] [08:31:43] <pieterh> maybe email will work better
[Wednesday 01 September 2010] [08:36:32] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Wednesday 01 September 2010] [08:36:32] Notice -NickServ- You are now identified for travlr.
[Wednesday 01 September 2010] [08:36:32] CTCP Received Version request from frigg.
[Wednesday 01 September 2010] [08:36:33] Topic The channel topic is “Welcome!”.
[Wednesday 01 September 2010] [08:36:40] Mode Channel modes: no messages from outside, topic protection
[Wednesday 01 September 2010] [08:36:40] Created This channel was created on 2010-02-05 10:44.
[Wednesday 01 September 2010] [08:39:25] <pieterh> sustrik: recv’d! thanks... git seems to have a problem working across slow links
[Wednesday 01 September 2010] [08:39:44] <pieterh> too much chatter or something... they aren’t using 0MQ....
[Wednesday 01 September 2010] [09:02:00] Quit poswald has left this server (Quit: poswald).
[Wednesday 01 September 2010] [09:10:29] <pieterh> sustrik: what’s your view of when 2.1.0 would be releasable?
[Wednesday 01 September 2010] [09:10:46] <pieterh> would it help if I stress tested it with some real applications?
[Wednesday 01 September 2010] [09:10:58] <pieterh> btw, it all build and passed the test suite fine, of course
[Wednesday 01 September 2010] [09:11:02] <pieterh>
*built
[Wednesday 01 September 2010] [09:11:48] <mikko> you can run tests in language bindings as well
[Wednesday 01 September 2010] [09:11:49] <guido_g> not a real suprise, the tests are really basic
[Wednesday 01 September 2010] [09:11:58] <mikko> they probably contain fair amount of tests
[Wednesday 01 September 2010] [09:12:58] <pieterh> gudio_g: i’d like to try breakage tests...
[Wednesday 01 September 2010] [09:13:13] <guido_g> nevertheless do we need good test coverage in the core
[Wednesday 01 September 2010] [09:13:27] <pieterh> yes, we can’t refer up the stack
[Wednesday 01 September 2010] [09:13:42] <pieterh> core can’t test using language bindings, this is up to the binding author/community
[Wednesday 01 September 2010] [09:13:55] <guido_g> pieterh: what does that mean?
[Wednesday 01 September 2010] [09:14:02] <guido_g> breakage tests
[Wednesday 01 September 2010] [09:14:17] <pieterh> well, stuff like send message, close, check that it’s delivered or not according to socket policy
[Wednesday 01 September 2010] [09:14:29] <pieterh> exceed HWM, check messages are delayed/discarded
[Wednesday 01 September 2010] [09:14:36] <pieterh> send invalid stuff, check 0MQ does not die
[Wednesday 01 September 2010] [09:14:50] <pieterh> do bad operations, invalid socket mixes, illegal sequences
[Wednesday 01 September 2010] [09:15:01] <pieterh> all the stuff normal apps do all the time... :-)
[Wednesday 01 September 2010] [09:15:23] <guido_g> ahh ok
[Wednesday 01 September 2010] [09:15:33] <guido_g> normal tests :)
[Wednesday 01 September 2010] [09:15:42] <pieterh> normal regression is of course also very useful
[Wednesday 01 September 2010] [09:15:45] <pieterh> “it still works”
[Wednesday 01 September 2010] [09:15:57] <pieterh> but “i can’t break it” is also valuable
[Wednesday 01 September 2010] [09:16:02] <guido_g> that you get with every build and make check
[Wednesday 01 September 2010] [09:16:18] <guido_g> the regression test
[Wednesday 01 September 2010] [09:16:25] <pieterh> yes
[Wednesday 01 September 2010] [09:16:37] <guido_g> ok, would be nice to have a list of tests to do
[Wednesday 01 September 2010] [09:16:49] <pieterh> it’s sometimes possible to collect test cases for issues
[Wednesday 01 September 2010] [09:16:51] <guido_g> i’ll go over it as time permits and write the test programs
[Wednesday 01 September 2010] [09:17:07] <pieterh> oh, sorry, i’m not trying to propose work... :-/
[Wednesday 01 September 2010] [09:17:20] <guido_g> it has to be done
[Wednesday 01 September 2010] [09:17:25] <pieterh> i’m more than happy to help though it would be in C not C++
[Wednesday 01 September 2010] [09:17:47] <pieterh> well, let’s start by collecting specs?
[Wednesday 01 September 2010] [09:17:53] <pieterh> sorry s/start/continue/
[Wednesday 01 September 2010] [09:18:04] <guido_g> yes
[Wednesday 01 September 2010] [09:18:19] <pieterh> perhaps this can help language binding authors make coherent test sets too
[Wednesday 01 September 2010] [09:18:29] <guido_g> to make it leight and easy to start with, a simple collection of ideas what to test would be ok i think
[Wednesday 01 September 2010] [09:18:30] <pieterh> i.e. standardized test set spec
[Wednesday 01 September 2010] [09:18:36] <pieterh> yes
[Wednesday 01 September 2010] [09:19:06] <pieterh> it would give users a tool for validating bindings
[Wednesday 01 September 2010] [09:19:16] <pieterh> “has test suite”
[Wednesday 01 September 2010] [09:20:12] <pieterh> actually it should then be written both in C++ and C
[Wednesday 01 September 2010] [09:20:25] <pieterh> i volunteer to make the C test suite then
[Wednesday 01 September 2010] [09:20:31] * pieterh opens his big mouth again
[Wednesday 01 September 2010] [09:22:25] <guido_g> :)
[Wednesday 01 September 2010] [09:22:38] <guido_g> i’d like to stick with c++ for now
[Wednesday 01 September 2010] [09:22:50] <guido_g> it’s hard enough to relearn it
[Wednesday 01 September 2010] [09:23:25] <pieterh> sure, so we will double up the test suite, that also gives people examples to compare?
[Wednesday 01 September 2010] [09:23:37] <pieterh> sustrik: do you have any problem with a C test suite in parallel to the C++ one?
[Wednesday 01 September 2010] [09:23:50] <pieterh> if it’s a stupid idea, i’m happy to retract it... :-)
[Wednesday 01 September 2010] [09:26:12] <guido_g> hehe
[Wednesday 01 September 2010] [09:26:24] * guido_g is leaving the office
[Wednesday 01 September 2010] [09:26:36] <guido_g> see you later
[Wednesday 01 September 2010] [09:27:19] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 01 September 2010] [09:29:46] <CIA-20> zeromq2: 03Martin Sustrik 07master * r651c1ad 10/ (src/ctx.cpp src/socket_base.cpp): sys transport introdced; inproc://log moved to sys://log - http://bit.ly/bmO6tM
[Wednesday 01 September 2010] [09:29:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0fe7d3c 10/ (builds/msvc/libzmq/libzmq.vcproj src/ctx.cpp): conflicts resolved -
http://bit.ly/aV1tLE[Wednesday 01 September 2010] [09:49:28] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [10:02:12] <guido_g> re
[Wednesday 01 September 2010] [10:03:13] <sustrik> pieterh: C test suite is OK
[Wednesday 01 September 2010] [10:03:31] <sustrik> but afaik the files have to have .cpp extension
[Wednesday 01 September 2010] [10:03:44] <sustrik> so that autotools know it should link c++ runtime with it
[Wednesday 01 September 2010] [10:04:11] <sustrik> (needed by 0mq library, in case of static linking)
[Wednesday 01 September 2010] [10:04:15] <pieterh> yes, indeed
[Wednesday 01 September 2010] [10:04:24] <pieterh> great
[Wednesday 01 September 2010] [10:04:42] <guido_g> why is the logging now a special transport?
[Wednesday 01 September 2010] [10:04:47] <pieterh> will have to use a naming convention to avoid conflicts, NP
[Wednesday 01 September 2010] [10:05:05] <sustrik> guido_g: two reasons
[Wednesday 01 September 2010] [10:05:39] <sustrik> 1. it’s a different beast, communicating with 0mq internals rather than communicating with a socket in a different thread
[Wednesday 01 September 2010] [10:05:50] <sustrik> 2. full backward compatibility
[Wednesday 01 September 2010] [10:06:14] <guido_g> ok
[Wednesday 01 September 2010] [10:06:49] <pieterh> sustrik: i see one problem with making it a separate transport
[Wednesday 01 September 2010] [10:06:54] <guido_g> how will users attach to this type of transport?
[Wednesday 01 September 2010] [10:07:08] <sustrik> guido_g: connect
[Wednesday 01 September 2010] [10:07:10] <guido_g> just a SUB w/ sys://log?
[Wednesday 01 September 2010] [10:07:16] <sustrik> pieterh: what problem?
[Wednesday 01 September 2010] [10:07:25] <sustrik> guido_g: yes
[Wednesday 01 September 2010] [10:07:31] <pieterh> take the syslog service as an analogy
[Wednesday 01 September 2010] [10:07:31] <guido_g> great! thanks
[Wednesday 01 September 2010] [10:07:40] <pieterh> it’s a bus, any component can participate
[Wednesday 01 September 2010] [10:08:08] <pieterh> ok, that’s not quite accurate but I mean you might want to allow arbitrary components to inject traffic into this
[Wednesday 01 September 2010] [10:08:19] <sustrik> i don’t :)
[Wednesday 01 September 2010] [10:08:24] <pieterh> and have it picked up by log clients without them needing to be changed
[Wednesday 01 September 2010] [10:08:25] <guido_g> :)
[Wednesday 01 September 2010] [10:08:46] <sustrik> what i mean is how would you scale that kind of thing?
[Wednesday 01 September 2010] [10:08:52] <sustrik> right now it’s easy
[Wednesday 01 September 2010] [10:09:06] <pieterh> it’s inproc or ipc, what’s the scaling issue?
[Wednesday 01 September 2010] [10:09:06] <sustrik> it’s constrained to the process
[Wednesday 01 September 2010] [10:09:17] <sustrik> the business logic
[Wednesday 01 September 2010] [10:09:24] <pieterh> attach device to sub, export as pub... np
[Wednesday 01 September 2010] [10:09:35] <sustrik> what you want with a business logic is a distributed logging service
[Wednesday 01 September 2010] [10:09:44] <guido_g> ack
[Wednesday 01 September 2010] [10:09:58] <guido_g> the way with a forwarder is ok
[Wednesday 01 September 2010] [10:10:07] <pieterh> i can play devil’s advocate here
[Wednesday 01 September 2010] [10:10:22] <sustrik> pieterh: i have no idea how you would implement it
[Wednesday 01 September 2010] [10:10:28] <pieterh> it looks like you are claiming a semantic difference in order to solve a naming clash
[Wednesday 01 September 2010] [10:10:40] <pieterh> it’s unclear what the value is of a new transport
[Wednesday 01 September 2010] [10:10:55] <pieterh> no clear problem you’re solving wrt to using inproc:
[Wednesday 01 September 2010] [10:10:55] <sustrik> you mean between sys & inproc or between 0mq logging and application logging?
[Wednesday 01 September 2010] [10:10:58] <pieterh> or ipc:
[Wednesday 01 September 2010] [10:12:22] <pieterh> sustrik: ... sorry, I don’t follow this thread
[Wednesday 01 September 2010] [10:12:36] <sustrik> pieterh> it looks like you are claiming a semantic difference in order to solve a naming clash
[Wednesday 01 September 2010] [10:12:45] <pieterh> ah, yes
[Wednesday 01 September 2010] [10:12:55] <pieterh> between sys: and inproc or ipc
[Wednesday 01 September 2010] [10:13:15] <pieterh> is sys: like inproc or like ipc?
[Wednesday 01 September 2010] [10:13:28] <sustrik> it’s different from both
[Wednesday 01 September 2010] [10:13:38] <sustrik> it has no peer actually
[Wednesday 01 September 2010] [10:13:55] <pieterh> yes, but can you connect to it from another process?
[Wednesday 01 September 2010] [10:13:59] <pieterh> is it an in or ex-process endpoint?
[Wednesday 01 September 2010] [10:14:12] <sustrik> it’s process-scoped
[Wednesday 01 September 2010] [10:14:19] <pieterh> so it’s like inproc
[Wednesday 01 September 2010] [10:14:30] <sustrik> sure, same scope
[Wednesday 01 September 2010] [10:14:33] <sustrik> different semantics
[Wednesday 01 September 2010] [10:15:01] <pieterh> now if i am being evil and minimalist i’ll ask what the value is in defining a totally new protocol instead of a protected namespace within inproc
[Wednesday 01 September 2010] [10:15:15] <pieterh> given that every new construct is something expensive to document, understand...
[Wednesday 01 September 2010] [10:15:45] <sustrik> same question: why not make ipc a separate namespace within tcp?
[Wednesday 01 September 2010] [10:15:51] <pieterh> but the semantics fit perfectly to an inproc SUB socket from the users’ POV
[Wednesday 01 September 2010] [10:16:09] <sustrik> no: there’s no peer socket
[Wednesday 01 September 2010] [10:16:23] <pieterh> that’s an implementation detail
[Wednesday 01 September 2010] [10:16:29] <pieterh> that’s behind the wall, and not relevant to users
[Wednesday 01 September 2010] [10:16:35] <pieterh> it looks like there is a peer socket
[Wednesday 01 September 2010] [10:16:38] <sustrik> ok, let me explain the internal mechanism of sys
[Wednesday 01 September 2010] [10:16:48] <pieterh> ah, hang on
[Wednesday 01 September 2010] [10:16:55] <pieterh> i’m at the API level
[Wednesday 01 September 2010] [10:17:04] <pieterh> internal mechanisms should not be relevant
[Wednesday 01 September 2010] [10:17:09] <guido_g> this is what sys: says imho, don’t expect the usual, this comming from the kernel
[Wednesday 01 September 2010] [10:17:24] <pieterh> but why?
[Wednesday 01 September 2010] [10:17:27] <pieterh> why the surprise?
[Wednesday 01 September 2010] [10:17:47] <guido_g> i can’t see any, sorry
[Wednesday 01 September 2010] [10:17:55] <pieterh> why is the kernel a ‘special’ component with its own transport that looks exactly like inproc: but has a different name?
[Wednesday 01 September 2010] [10:17:57] <pieterh> it’s surprising
[Wednesday 01 September 2010] [10:18:03] <sustrik> it’s administration/business logic distinction
[Wednesday 01 September 2010] [10:18:10] <sustrik> inproc carries business logic
[Wednesday 01 September 2010] [10:18:17] <sustrik> sys carries admin data
[Wednesday 01 September 2010] [10:18:18] <pieterh> not necessarily
[Wednesday 01 September 2010] [10:18:24] <sustrik> necessarily
[Wednesday 01 September 2010] [10:18:24] <pieterh> as soon as i make a framework over 0MQ
[Wednesday 01 September 2010] [10:18:30] <pieterh> then inproc carries admin data
[Wednesday 01 September 2010] [10:18:39] <guido_g> for an application using the core
[Wednesday 01 September 2010] [10:18:50] <guido_g> not the mq core itself
[Wednesday 01 September 2010] [10:18:58] <pieterh> ok... perhaps it would help if there was some significant semantic difference between admin data and business data
[Wednesday 01 September 2010] [10:19:01] <pieterh> such as priority
[Wednesday 01 September 2010] [10:19:02] <guido_g> i see sys: as the core namespace
[Wednesday 01 September 2010] [10:19:07] <pieterh> or blocking/ error handling
[Wednesday 01 September 2010] [10:19:19] <pieterh> guido_g: as a namespace this is a lousy solution IMO
[Wednesday 01 September 2010] [10:19:36] <pieterh> namespaces should be orthogonal to transports
[Wednesday 01 September 2010] [10:19:36] <guido_g> pieterh: why?
[Wednesday 01 September 2010] [10:19:57] <pieterh> well, this makes it impossible to run a separate logger to inspect a running process
[Wednesday 01 September 2010] [10:20:13] <pieterh> every single application needs a special thread that exports the sys: data over ipc: or tcp:
[Wednesday 01 September 2010] [10:20:17] <sustrik> the main problem i have with the whole log thing is that people are now going to use it for their business logic and thus break the patterns
[Wednesday 01 September 2010] [10:20:26] <sustrik> if so, i’ll rather remove it completely
[Wednesday 01 September 2010] [10:20:28] <pieterh> sustrik: no, you can prevent that easily
[Wednesday 01 September 2010] [10:20:36] <sustrik> ?
[Wednesday 01 September 2010] [10:20:36] <guido_g> sustrik: hold it, we need it!
[Wednesday 01 September 2010] [10:20:38] <pieterh> you can prevent that trivially
[Wednesday 01 September 2010] [10:20:43] <pieterh> use a protected name space
[Wednesday 01 September 2010] [10:20:56] <guido_g> like sys: ? :)
[Wednesday 01 September 2010] [10:21:00] <pieterh> recall in amqp how exchanges beginning with amq. were protected
[Wednesday 01 September 2010] [10:21:10] <pieterh> transports are not namespaces... :-(
[Wednesday 01 September 2010] [10:21:20] <guido_g> sure
[Wednesday 01 September 2010] [10:21:27] <sustrik> what i mean is people are now going to ask me to report disconnections via sys
[Wednesday 01 September 2010] [10:21:29] <pieterh> inproc://zmq/log and ipc://zmq/log would be perfect
[Wednesday 01 September 2010] [10:21:31] <pieterh> allows me to start separate logger
[Wednesday 01 September 2010] [10:21:38] <sustrik> so that they can handle each connection separately
[Wednesday 01 September 2010] [10:21:59] <pieterh> perhaps
[Wednesday 01 September 2010] [10:22:08] <sustrik> to prevent that, the admin part should be as much separated from the business part as possible
[Wednesday 01 September 2010] [10:22:09] <pieterh> that’s a use case but it’s possibly not the right solution
[Wednesday 01 September 2010] [10:22:18] <pieterh> hmm
[Wednesday 01 September 2010] [10:22:22] <sustrik> different API would be ideal
[Wednesday 01 September 2010] [10:22:36] <guido_g> but the application developer doe not know what makes sys: work technically, so the obvious similarity to inproc might be wrong
[Wednesday 01 September 2010] [10:22:36] <sustrik> but that’s too much work
[Wednesday 01 September 2010] [10:22:38] <pieterh> sustrik: do you see the value in allowing this namespace to work over both ipc and inproc?
[Wednesday 01 September 2010] [10:22:48] <sustrik> no
[Wednesday 01 September 2010] [10:23:07] <sustrik> it’s just adminstration stuff
[Wednesday 01 September 2010] [10:23:37] <pieterh> i don’t think logging should be mixed with out of band control of sockets
[Wednesday 01 September 2010] [10:23:45] <pieterh> that’s a different issue
[Wednesday 01 September 2010] [10:23:47] <sustrik> right
[Wednesday 01 September 2010] [10:23:55] <sustrik> but it’s an obvious trap
[Wednesday 01 September 2010] [10:24:11] <pieterh> avoidable trap
[Wednesday 01 September 2010] [10:24:13] <sustrik> i think someone have already suggested something like that
[Wednesday 01 September 2010] [10:24:19] <pieterh> yes
[Wednesday 01 September 2010] [10:24:27] <sustrik> i can avoid it by saying “NO”
[Wednesday 01 September 2010] [10:24:36] <sustrik> as i am core maintainer atm
[Wednesday 01 September 2010] [10:24:46] <pieterh> indeed
[Wednesday 01 September 2010] [10:24:53] <pieterh> but that’s not going away, your veto
[Wednesday 01 September 2010] [10:24:57] <sustrik> no mixing of business logic and administration
[Wednesday 01 September 2010] [10:25:04] <pieterh> anyone can fork 0MQ and add whatever they like
[Wednesday 01 September 2010] [10:25:10] <pieterh> but it won’t be the official version
[Wednesday 01 September 2010] [10:25:16] <sustrik> sure, that’s ok
[Wednesday 01 September 2010] [10:25:19] <pieterh> unless and until you vote it in
[Wednesday 01 September 2010] [10:25:27] <pieterh> no exceptions to this, period.
[Wednesday 01 September 2010] [10:25:58] <guido_g> are we still at the logging thing or do i have to move myself to a safe place?
[Wednesday 01 September 2010] [10:26:00] <sustrik> good, so unless i hear a good argument for mixing business logic and maitenence/debugging stuff, i would keep it apart
[Wednesday 01 September 2010] [10:26:18] <sustrik> it’s logging still :)
[Wednesday 01 September 2010] [10:26:27] <pieterh> sustrik: i’d like, really love, to be able to do logging from a separate process
[Wednesday 01 September 2010] [10:26:42] <pieterh> otherwise every application becomes N x more complex
[Wednesday 01 September 2010] [10:26:42] <sustrik> write an app that does that
[Wednesday 01 September 2010] [10:26:43] <sustrik> np
[Wednesday 01 September 2010] [10:26:50] <sustrik> 5 lines of code
[Wednesday 01 September 2010] [10:27:14] <pieterh> you said it was in process
[Wednesday 01 September 2010] [10:27:25] <pieterh> did I misunderstand you?
[Wednesday 01 September 2010] [10:27:35] <sustrik> you can create a device that would read from sys and republish on tcp
[Wednesday 01 September 2010] [10:27:44] <sustrik> you can even use forwarder
[Wednesday 01 September 2010] [10:28:19] <sustrik> when you do that you are effectively converting 0mq admin data into business logic data
[Wednesday 01 September 2010] [10:28:20] <sustrik> which is ok
[Wednesday 01 September 2010] [10:28:30] <sustrik> but at 0mq level i need them separated
[Wednesday 01 September 2010] [10:28:36] <sustrik> to avoid contamination
[Wednesday 01 September 2010] [10:28:50] <guido_g> if anyone can access the internal data flow w/o authorization it would be a nightmare
[Wednesday 01 September 2010] [10:29:05] <sustrik> you mean the logs?
[Wednesday 01 September 2010] [10:29:09] <guido_g> yes
[Wednesday 01 September 2010] [10:29:16] <sustrik> :(
[Wednesday 01 September 2010] [10:29:26] <sustrik> no authorisation as for now
[Wednesday 01 September 2010] [10:29:33] <sustrik> otoh
[Wednesday 01 September 2010] [10:29:38] <sustrik> it’s in-process only
[Wednesday 01 September 2010] [10:29:49] <sustrik> so nobody from outside can read from sys
[Wednesday 01 September 2010] [10:29:52] <guido_g> given the “one forwarder per program” model, the program decides on what to publish and where etc.
[Wednesday 01 September 2010] [10:30:31] <guido_g> sustrik: sure, but as far as i understood, pieterh wanted access from an external process w/o a forwarder inbetween
[Wednesday 01 September 2010] [10:30:47] <sustrik> yes, that was the original discussion
[Wednesday 01 September 2010] [10:31:10] <sustrik> actually it would be almost impossible from implementation point of view
[Wednesday 01 September 2010] [10:31:20] <sustrik> that’s black magic going on in the background :)
[Wednesday 01 September 2010] [10:31:21] <guido_g> details! :)
[Wednesday 01 September 2010] [10:31:44] <sustrik> well, sys is technically different from all the other sockets
[Wednesday 01 September 2010] [10:31:56] <sustrik> maybe pgm excluded
[Wednesday 01 September 2010] [10:32:02] <sustrik> becasue it’s a real bus
[Wednesday 01 September 2010] [10:32:08] <sustrik> ie. multiple writers
[Wednesday 01 September 2010] [10:32:13] <sustrik> multiple readers
[Wednesday 01 September 2010] [10:32:19] <sustrik> no device in the middle
[Wednesday 01 September 2010] [10:32:28] <guido_g> as i said “<guido_g> but the application developer doe not know what makes sys: work technically, so the obvious similarity to inproc might be wrong”
[Wednesday 01 September 2010] [10:32:56] <guido_g> therefore the sys: transport makes sense in my eyes
[Wednesday 01 September 2010] [10:33:09] <sustrik> yes, that was my argument as well
[Wednesday 01 September 2010] [10:33:14] <guido_g> good
[Wednesday 01 September 2010] [10:33:33] <guido_g> because i really like the logging
[Wednesday 01 September 2010] [10:33:48] <sustrik> it’s good but can be dangerous if misused
[Wednesday 01 September 2010] [10:34:04] <guido_g> mq is notoriously weak on giving information on what’s going on
[Wednesday 01 September 2010] [10:34:14] <sustrik> :|
[Wednesday 01 September 2010] [10:34:23] <guido_g> that’s a thing nearly every library has
[Wednesday 01 September 2010] [10:34:35] <sustrik> right
[Wednesday 01 September 2010] [10:34:52] <sustrik> that’s because it’s much more async that probably any other library around
[Wednesday 01 September 2010] [10:35:02] <sustrik> zillions of moving parts
[Wednesday 01 September 2010] [10:35:09] <sustrik> even without proper synchronisation
[Wednesday 01 September 2010] [10:35:12] <guido_g> and for real distributed applications there is rule 0: log everything, all the time
[Wednesday 01 September 2010] [10:35:21] <sustrik> right
[Wednesday 01 September 2010] [10:35:27] <sustrik> so, this should solve it
[Wednesday 01 September 2010] [10:35:32] <sustrik> hopefully
[Wednesday 01 September 2010] [10:35:47] <guido_g> it’s at least a clean interface
[Wednesday 01 September 2010] [10:36:01] <pieterh> sustrik: makes sense but it’s a shame that we’d need a dedicated thread in every app to export log data
[Wednesday 01 September 2010] [10:36:10] <pieterh> still, this can be extended over time
[Wednesday 01 September 2010] [10:36:23] <pieterh> minimalism demands I shut up now :-)
[Wednesday 01 September 2010] [10:36:41] <pieterh> though a new question comes to mind...
[Wednesday 01 September 2010] [10:36:56] <pieterh> if a bus transport is useful for 0MQ internally
[Wednesday 01 September 2010] [10:37:01] <pieterh> surely it is also useful for applications as well
[Wednesday 01 September 2010] [10:37:07] <pieterh> :-)
[Wednesday 01 September 2010] [10:37:09] <sustrik> it doesn’t scale
[Wednesday 01 September 2010] [10:37:17] <sustrik> it can work on level on 1 process
[Wednesday 01 September 2010] [10:37:24] <sustrik> even on LAN
[Wednesday 01 September 2010] [10:37:35] <sustrik> once you try to stretch it further it breaks
[Wednesday 01 September 2010] [10:38:00] <sustrik> the number of transfers grows quadratically with increasding number of components
[Wednesday 01 September 2010] [10:38:38] <sustrik> pieterh: ad dedicated thread
[Wednesday 01 September 2010] [10:38:42] <pieterh> sustrik: as an inproc bus, I meant
[Wednesday 01 September 2010] [10:38:45] <sustrik> yes, that’s why it’s a bus
[Wednesday 01 September 2010] [10:38:46] <pieterh> bus://
[Wednesday 01 September 2010] [10:38:58] <sustrik> to avoid speacial thread
[Wednesday 01 September 2010] [10:39:14] <sustrik> yes, can be
[Wednesday 01 September 2010] [10:39:37] <sustrik> but i like that sys is explicit abour “don’t pass your business logic through this”
[Wednesday 01 September 2010] [10:39:48] <pieterh> for some reason ‘magic system stuff you use here’ seems not ideal
[Wednesday 01 September 2010] [10:40:16] <pieterh> it’s like the core gets to use magic stuff the apps don’t get
[Wednesday 01 September 2010] [10:40:28] <sustrik> yes
[Wednesday 01 September 2010] [10:40:31] <guido_g> that’s why it’s the core
[Wednesday 01 September 2010] [10:40:37] <sustrik> because core doesn’t have to scale
[Wednesday 01 September 2010] [10:40:42] <sustrik> applications do
[Wednesday 01 September 2010] [10:41:36] <pieterh> imo you will get people demanding access to this transport for their essential internal signalling
[Wednesday 01 September 2010] [10:41:51] <pieterh> but i have no proof, just baseless speculation
[Wednesday 01 September 2010] [10:42:07] <sustrik> i expect it will be so
[Wednesday 01 September 2010] [10:42:19] <sustrik> thus i want it be named something really scary
[Wednesday 01 September 2010] [10:42:30] <guido_g> but the line is drawn “sorry, no access for you here”
[Wednesday 01 September 2010] [10:42:34] <pieterh> e.g. “how do I do a clean shutdown”... answer: signal every thread
[Wednesday 01 September 2010] [10:42:44] <guido_g> as with system calls into the kernel
[Wednesday 01 September 2010] [10:42:45] <pieterh> bah. that was the idea with PAIR as well
[Wednesday 01 September 2010] [10:42:49] <sustrik> that’s baked into 0mq as well
[Wednesday 01 September 2010] [10:42:52] <sustrik> ETERM
[Wednesday 01 September 2010] [10:42:55] <pieterh> but it turns out we really need PAIR for some use cases
[Wednesday 01 September 2010] [10:43:15] <sustrik> ok, the use case for pair:
[Wednesday 01 September 2010] [10:43:18] <pieterh> yeah, i know, it’s a bad example
[Wednesday 01 September 2010] [10:43:26] <sustrik> you have an application that does accounting and HR
[Wednesday 01 September 2010] [10:43:34] <sustrik> you want to split it into two applications
[Wednesday 01 September 2010] [10:43:37] <sustrik> one does accounting
[Wednesday 01 September 2010] [10:43:40] <sustrik> other does HR
[Wednesday 01 September 2010] [10:43:52] <sustrik> they have a PAIR socket between them
[Wednesday 01 September 2010] [10:44:10] <sustrik> because each of the peer is singleton
[Wednesday 01 September 2010] [10:44:16] <sustrik> and doesn’t scale
[Wednesday 01 September 2010] [10:44:23] <pieterh> my use case for PAIR is just interthread signalling
[Wednesday 01 September 2010] [10:44:32] <sustrik> hm
[Wednesday 01 September 2010] [10:44:37] <sustrik> what use case?
[Wednesday 01 September 2010] [10:44:52] <pieterh> i explained it in the guide, end of chapter 2
[Wednesday 01 September 2010] [10:45:00] <pieterh> thread a wants to signal to thread b that it’s done
[Wednesday 01 September 2010] [10:45:05] <pieterh> no scaling required
[Wednesday 01 September 2010] [10:45:10] <sustrik> let me see
[Wednesday 01 September 2010] [10:45:14] <pieterh> no reconnection required
[Wednesday 01 September 2010] [10:45:25] <pieterh> PAIR gives the best semantics afaics
[Wednesday 01 September 2010] [10:47:48] * sustrik is looking at the program
[Wednesday 01 September 2010] [10:48:00] <sustrik> this one: “Thread Coordination” ?
[Wednesday 01 September 2010] [10:51:38] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Wednesday 01 September 2010] [10:51:55] <sustrik> well, there’s no business logic in the example
[Wednesday 01 September 2010] [10:52:09] <sustrik> so it’s hard to say what’s the right socket types to use
[Wednesday 01 September 2010] [10:52:39] <sustrik> it the three components are by definition singletons, then PAIR is appropriate
[Wednesday 01 September 2010] [10:52:54] <sustrik> it say step2 can be parallelised the pipeline would be better
[Wednesday 01 September 2010] [10:53:10] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [10:53:49] <sustrik> the problem i have with pair is that it solves a problem (the accounting vs. HR problem)
[Wednesday 01 September 2010] [10:53:54] <sustrik> but it doesn’t scale
[Wednesday 01 September 2010] [10:54:29] <sustrik> by throwing more boxes on your accounting/HR application you won’t get more power
[Wednesday 01 September 2010] [10:54:43] <sustrik> even thought you paired the two by a PAIR socket
[Wednesday 01 September 2010] [11:17:37] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [11:17:48] Nick
pieterh_ is now known as pieterh.
[Wednesday 01 September 2010] [11:22:08] Quit gebi has left this server (Ping timeout: 276 seconds).
[Wednesday 01 September 2010] [11:29:51] <guido_g> sustrik: latest master shows poll problem again
[Wednesday 01 September 2010] [11:38:58] Quit mrm2m has left this server (Quit: Leaving.).
[Wednesday 01 September 2010] [12:11:55] <ModusPwnens> hello
[Wednesday 01 September 2010] [12:18:26] <ModusPwnens> i have a question about the zmq_msg_size function
[Wednesday 01 September 2010] [12:26:24] <cremes> ModusPwnens: go ahead and ask
[Wednesday 01 September 2010] [12:26:33] <cremes> if someone knows the answer, they’ll respond
[Wednesday 01 September 2010] [12:26:38] <ModusPwnens> ok
[Wednesday 01 September 2010] [12:26:56] <ModusPwnens> so as i have said before, I am using google protocl buffers to encode messages and send them
[Wednesday 01 September 2010] [12:27:29] <ModusPwnens> and on the clientsize, I do message.size(), where message is the size of the object encoded by google protocol buffers
[Wednesday 01 September 2010] [12:27:33] <ModusPwnens> however, on the server side
[Wednesday 01 September 2010] [12:27:41] <ModusPwnens> I use the zmq_msg_size function
[Wednesday 01 September 2010] [12:27:55] <ModusPwnens> but no matter how large the message actually is, zmq_msg_size always gives 36 bytes
[Wednesday 01 September 2010] [12:28:14] <ModusPwnens> I am wondering not only why that is, but how to get the actual size of the payload of the message
[Wednesday 01 September 2010] [12:28:21] <cremes> ok
[Wednesday 01 September 2010] [12:28:48] <cremes> it sounds to me like your client is not allocating the message correctly
[Wednesday 01 September 2010] [12:29:11] <cremes> do you have a code snippet showing how you are allocating the zmq_msg_t and assigning your protocol buffer to it?
[Wednesday 01 September 2010] [12:29:36] <ModusPwnens> rc = zmq_msg_init_data(&msg, cstr, message.size(), NULL, NULL);
[Wednesday 01 September 2010] [12:29:41] <ModusPwnens> something like that?>
[Wednesday 01 September 2010] [12:30:43] <cremes> hold on a sec...
[Wednesday 01 September 2010] [12:30:51] <ModusPwnens> k
[Wednesday 01 September 2010] [12:32:32] <cremes> the variable ‘cstr’ points to the data; are you sure its size is the same as message.size() ?
[Wednesday 01 September 2010] [12:32:47] <Steve-o> shouldn’t it be message.ByteSize() in Google protobufs?
[Wednesday 01 September 2010] [12:33:26] * cremes thinks that is a good question too since i am unfamiliar with that api
[Wednesday 01 September 2010] [12:34:08] <ModusPwnens> hmm, that’s a good point. I wasn’t aware that function existed..
[Wednesday 01 September 2010] [12:34:26] <ModusPwnens> but in any case, that doesn’t seem like it would solve my problem.
[Wednesday 01 September 2010] [12:35:03] <cremes> does the server always receive packets of length 36 exactly? if so, the client must be producing messages of that length
[Wednesday 01 September 2010] [12:35:25] <cremes> focus on the client side for now... something is likely wrong there
[Wednesday 01 September 2010] [12:35:59] <cremes> on the server, are the bytes received garbage? try printing the data the client sends and comparing it to the data the server receives
[Wednesday 01 September 2010] [12:36:10] <ModusPwnens> It seems so, but when i print the value of message.size() in the client, it prints outs omething much larger
[Wednesday 01 September 2010] [12:36:16] <ModusPwnens> and the bytes received are not garbage
[Wednesday 01 September 2010] [12:36:21] <ModusPwnens> i have verified the message coming through is valid
[Wednesday 01 September 2010] [12:37:17] <cremes> why not try: rc = zmq_msg_init_data(&msg, cstr, strlen(cstr), NULL, NULL);
[Wednesday 01 September 2010] [12:37:48] <cremes> i don’t understand the relationship between cstr and message.size()
[Wednesday 01 September 2010] [12:38:21] <ModusPwnens> Hmm, i changed that for a reason...
[Wednesday 01 September 2010] [12:38:32] <ModusPwnens> I think it’s because the zmq syntax doesn’t want a string there or something
[Wednesday 01 September 2010] [12:38:41] <ModusPwnens> so I had to convert a string into a c-style string
[Wednesday 01 September 2010] [12:38:46] <ModusPwnens> and thats what cstr is
[Wednesday 01 September 2010] [12:39:08] <cremes> ah ha... that is your problem
[Wednesday 01 September 2010] [12:39:19] <ModusPwnens> it is?
[Wednesday 01 September 2010] [12:39:20] <cremes> c style strings are terminated by a NUL 0
[Wednesday 01 September 2010] [12:39:50] <ModusPwnens> Yeah. Does that interfere with zeromq somehow?
[Wednesday 01 September 2010] [12:39:59] <cremes> hmmm, wait, i got ahead of myself there...
[Wednesday 01 September 2010] [12:40:14] <cremes> i’m going to think out loud
[Wednesday 01 September 2010] [12:40:17] <ModusPwnens> ok
[Wednesday 01 September 2010] [12:40:31] <cremes> so a c style string is really just a char* or a pointer to a byte buffer
[Wednesday 01 September 2010] [12:40:45] <ModusPwnens> correct
[Wednesday 01 September 2010] [12:40:48] <ModusPwnens> oh right
[Wednesday 01 September 2010] [12:40:51] <cremes> you need to pass a byte buffer pointing to the beginning of your encoded message
[Wednesday 01 September 2010] [12:40:52] <ModusPwnens> sorry you’re thinking out loud >_<
[Wednesday 01 September 2010] [12:41:06] <cremes> is the cstr pointing to that?
[Wednesday 01 September 2010] [12:41:13] <ModusPwnens> yeah
[Wednesday 01 September 2010] [12:41:28] <ModusPwnens> and then i strcpy the message into cstr
[Wednesday 01 September 2010] [12:41:29] <cremes> e.g. cstr = (char*) message.encode().get_pointer()
[Wednesday 01 September 2010] [12:41:30] <ModusPwnens> beforehand
[Wednesday 01 September 2010] [12:41:48] <ModusPwnens> yeah something like that
[Wednesday 01 September 2010] [12:41:49] <cremes> any embedded NULs will terminate the string copy
[Wednesday 01 September 2010] [12:42:06] <cremes> so you may only be copying 36 bytes even though your byte buffer is larger
[Wednesday 01 September 2010] [12:42:10] <cremes> use memcpy instead
[Wednesday 01 September 2010] [12:42:20] <cremes> does that sound reasonable?
[Wednesday 01 September 2010] [12:42:34] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [12:42:38] <ModusPwnens> but..wouldn’t that reflect on the serverside as far as displaying the message?
[Wednesday 01 September 2010] [12:42:47] <ModusPwnens> i.e. it wouldn’t display the whole thing?
[Wednesday 01 September 2010] [12:43:10] <ModusPwnens> furthermore
[Wednesday 01 September 2010] [12:43:18] <ModusPwnens> embedded nuls will terminate the string copy
[Wednesday 01 September 2010] [12:43:29] <ModusPwnens> but doesnt’ that only apply to the encoded message that is being copied?
[Wednesday 01 September 2010] [12:43:47] <cremes> ModusPwnens: you are going to have to show more code, i think
[Wednesday 01 September 2010] [12:44:15] <ModusPwnens> ok i will just copy the entire function into a pastebin
[Wednesday 01 September 2010] [12:44:19] <cremes> there are lots of folks successfully slinging around large strings and large byte buffers so there is likely an issue in your code
[Wednesday 01 September 2010] [12:44:27] <cremes> cool
[Wednesday 01 September 2010] [12:44:32] <ModusPwnens> That is definitely a possibility
[Wednesday 01 September 2010] [12:44:49] <cremes> you are using 2.0.8 release, yes? or the 2.1 master branch?
[Wednesday 01 September 2010] [12:44:58] <ModusPwnens> 2.0.7
[Wednesday 01 September 2010] [12:45:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r99ddfa7 10/ (builds/msvc/platform.hpp configure.in): maint: will become 2.0.9 -
http://bit.ly/dgicJb[Wednesday 01 September 2010] [12:45:26] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47aaf10 10/ :
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: * maint:
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: maint: will become 2.0.9
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: Conflicts:
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: builds/msvc/platform.hpp
[Wednesday 01 September 2010] [12:45:28] <CIA-20> zeromq2: 03Mikael Helbo Kjr 07master * r59315eb 10/ src/select.cpp :
[Wednesday 01 September 2010] [12:45:29] <CIA-20> zeromq2: Erasure of retired fd’s in select.cpp causes an assertion in MSVC 2008 STL
[Wednesday 01 September 2010] [12:45:29] <CIA-20> zeromq2: I was hitting an issue with an SCL enabled STL library in connection with the
[Wednesday 01 September 2010] [12:45:30] <CIA-20> zeromq2: way select_t::loop was erasing retired fd’s. The problem as identified by the
[Wednesday 01 September 2010] [12:45:31] <CIA-20> zeromq2: SCL assertion was that by the time the iterator given to the erase method was
[Wednesday 01 September 2010] [12:45:31] <CIA-20> zeromq2: called it was considered invalid by the library. I am not sure this isn’t just
[Wednesday 01 September 2010] [12:45:32] <CIA-20> zeromq2: a “quirk” of the MSVC STL library as the other code looks valid to me as well. -
http://bit.ly/dzUsD3[Wednesday 01 September 2010] [12:45:32] <CIA-20> zeromq2: 03Martin Sustrik 07master * ra81a373 10/ src/select.cpp :
[Wednesday 01 September 2010] [12:45:33] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 01 September 2010] [12:45:55] <cremes> 2.0.7 should be okay too; i don’t recall any data truncation bugs for any sockets
[Wednesday 01 September 2010] [12:46:35] <ModusPwnens> if i am doing anything stupid in general
[Wednesday 01 September 2010] [12:46:37] <ModusPwnens> please let me know
[Wednesday 01 September 2010] [12:46:59] <ModusPwnens> I am actually only a stupid so im not as experienced as most people here probably are..
[Wednesday 01 September 2010] [12:47:02] <ModusPwnens> wow
[Wednesday 01 September 2010] [12:47:07] <ModusPwnens> i meant student
[Wednesday 01 September 2010] [12:47:35] <cremes> line 36 confuses me...
[Wednesday 01 September 2010] [12:47:41] <ModusPwnens> oh yeah ignore that
[Wednesday 01 September 2010] [12:47:54] <cremes> is that supposed to be commented out?
[Wednesday 01 September 2010] [12:48:27] <cremes> because it is overwriting the size of the msg on the prior line
[Wednesday 01 September 2010] [12:48:37] <ModusPwnens> I think so. We put that in for debugging purposes.
[Wednesday 01 September 2010] [12:48:41] <cremes> with 36 bytes which matches up with how many you are receiving on the server side
[Wednesday 01 September 2010] [12:49:00] <cremes> well, that’s a bug
[Wednesday 01 September 2010] [12:49:06] <cremes> remove that and test again
[Wednesday 01 September 2010] [12:49:13] <ModusPwnens> sigh i hate asking stupid questions
[Wednesday 01 September 2010] [12:51:05] Quit plq has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [12:51:06] <cremes> don’t worry about it; you’ll be repaying the favor here in a few short weeks
[Wednesday 01 September 2010] [12:51:34] <ModusPwnens> ahhhhh gosh darn it
[Wednesday 01 September 2010] [12:51:35] <ModusPwnens> that was it
[Wednesday 01 September 2010] [12:51:41] <ModusPwnens> I’m sorry for wasting your time.
[Wednesday 01 September 2010] [12:51:45] Join jond has joined this channel (
~jon@87.74.16.24).
[Wednesday 01 September 2010] [12:51:45] <ModusPwnens> thanks for your help tho
[Wednesday 01 September 2010] [12:51:46] <cremes> btw, line 34 looks to me like a bug waiting to happen
[Wednesday 01 September 2010] [12:51:51] <cremes> i would use memcpy
[Wednesday 01 September 2010] [12:52:02] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [12:52:27] <ModusPwnens> because of the nul string thing?
[Wednesday 01 September 2010] [12:52:49] <cremes> right
[Wednesday 01 September 2010] [12:53:05] <cremes> memcpy(cstr, message.c_str(), message.size())
[Wednesday 01 September 2010] [12:53:11] <ModusPwnens> Ok, i will change it.
[Wednesday 01 September 2010] [12:53:13] <cremes> or use strncpy
[Wednesday 01 September 2010] [12:53:25] <cremes> but memcpy is likely quicker since it isn’t looking for NULs
[Wednesday 01 September 2010] [12:53:48] <cremes> ModusPwnens: and all this from a Ruby guy who hasn’t used C in anger for many years... :)
[Wednesday 01 September 2010] [12:54:10] <ModusPwnens> heh, well it looks like you still got it then.
[Wednesday 01 September 2010] [12:54:36] <ModusPwnens> oh, and i have another question this time
[Wednesday 01 September 2010] [12:54:48] <ModusPwnens> for the send and receive topology
[Wednesday 01 September 2010] [12:55:10] <ModusPwnens> is there a flag you can set or something that prevents the send and receive messages from blocking?
[Wednesday 01 September 2010] [12:55:54] <cremes> yes
[Wednesday 01 September 2010] [12:56:37] <cremes> btw, when using PUB sockets, it’s a good idea to put your topic in a separate message part
[Wednesday 01 September 2010] [12:57:00] <ModusPwnens> what do you mean?
[Wednesday 01 September 2010] [12:57:12] <cremes> the example code you pastied showed that you are sending the AddressBook as the first message part, so subscribers will match on it
[Wednesday 01 September 2010] [12:57:30] <cremes> i’ll pastie some pseudo-code showing what i mean
[Wednesday 01 September 2010] [12:57:39] <ModusPwnens> ok.
[Wednesday 01 September 2010] [12:58:46] <ModusPwnens> and as far as the no blocking, we want to use it so we can constantly send messages and then receive a message every once in a while, or vice-versa. But would it be better to just use multiple sockets and a publish-subscribe topology instead?
[Wednesday 01 September 2010] [12:58:47] <jond> cremes: if it’s std::string why not use the copy method std::string::copy?
[Wednesday 01 September 2010] [12:59:21] <mikko> do you actually need a copy there?
[Wednesday 01 September 2010] [12:59:58] <ModusPwnens> well i need to store the message I am sending in that string, so I would need to copy it in, right?
[Wednesday 01 September 2010] [13:01:33] <cremes> jond: i don’t know; i’m not a C++ guy but memcpy is pretty easy to figure out for me
[Wednesday 01 September 2010] [13:02:01] <cremes> mikko: you probably need the copy unless you can guarantee the buffer won’t go away until the message is actually sent
[Wednesday 01 September 2010] [13:02:46] <ModusPwnens> Hmm, that makes sense.
[Wednesday 01 September 2010] [13:02:52] <Steve-o> Does it really need to be a string? I used buf.SerializeToArray() and buf.ParseFromArray().
[Wednesday 01 September 2010] [13:03:56] <cremes> ModusPwnens: by sending a topic, your SUB sockets can use it to filter out unwanted messages
[Wednesday 01 September 2010] [13:04:27] <ModusPwnens> Yes that makes sense. I can see the usefulness of that.
[Wednesday 01 September 2010] [13:04:34] <ModusPwnens> And steve-o, you make a good point
[Wednesday 01 September 2010] [13:04:37] <jond> cremes: ok, there is a method on string, so you could do message.copy(cstr,message.size()), but the code id as you say mainly c
[Wednesday 01 September 2010] [13:04:54] <mikko> ModusPwnens: is that C++ or C ?
[Wednesday 01 September 2010] [13:05:06] <cremes> jond: i’m a ruby guy so take my C suggestions with a grain of salt ;)
[Wednesday 01 September 2010] [13:05:13] <ModusPwnens> well, I am using visual studio C++, but I am actually using the C bindings
[Wednesday 01 September 2010] [13:05:23] <mikko> cremes: yeah, true
[Wednesday 01 September 2010] [13:05:28] <mikko> on the earlier
[Wednesday 01 September 2010] [13:05:28] <ModusPwnens> mostly because I was having trouble with the C++ ones
[Wednesday 01 September 2010] [13:06:28] <ModusPwnens> and steve-o, that would probably be a better/more efficient way of doing this, right?
[Wednesday 01 September 2010] [13:06:43] <ModusPwnens> then I wouldn’t have to do all the c-string manipulation
[Wednesday 01 September 2010] [13:07:10] <Steve-o> it should be the best as of v2.1, but I think Google has changed their api a few times since
[Wednesday 01 September 2010] [13:09:06] <Steve-o> noting ByteSize() is only valid after you have actually filled in values for protobufs, it has caught me a few times
[Wednesday 01 September 2010] [13:09:24] <ModusPwnens> you mean after you have encoded a message?
[Wednesday 01 September 2010] [13:10:28] <Steve-o> I needed the message size before serializing
[Wednesday 01 September 2010] [13:11:28] <ModusPwnens> hmm
[Wednesday 01 September 2010] [13:11:32] <ModusPwnens> for the serializetoarray method
[Wednesday 01 September 2010] [13:11:49] Join gebi has joined this channel (
~gebi@84.119.80.149).
[Wednesday 01 September 2010] [13:12:10] <ModusPwnens> you would need to get the size of the data you are serializing before you can serialize it..?
[Wednesday 01 September 2010] [13:13:54] <Steve-o> For my code I need to reserve space in a socket buffer for the serialized form, so I need the size. Obviously usage case varies for everyone.
[Wednesday 01 September 2010] [13:14:43] <ModusPwnens> You mean the size is always known beforehand? What if you don’t know the size before, can you still use that?
[Wednesday 01 September 2010] [13:15:14] <Steve-o> probably if you know the protobuf object is always going to be smaller than your buffer
[Wednesday 01 September 2010] [13:15:45] <ModusPwnens> Hmm. okay.
[Wednesday 01 September 2010] [13:15:57] <Steve-o> it’s a bit safer to check the size before hand, much like vasprintf()
[Wednesday 01 September 2010] [13:19:44] <ModusPwnens> Yeah that makes sense.
[Wednesday 01 September 2010] [13:19:49] <ModusPwnens> Ok, thanks for your help guys!
[Wednesday 01 September 2010] [13:20:17] <ModusPwnens> I will probably be back again in a few days with something else..hopefully something less stupid.
[Wednesday 01 September 2010] [13:53:25] Quit xla has left this server (Quit: leaving).
[Wednesday 01 September 2010] [13:55:34] <pieterh> folks: if anyone wants to review
http://www.zeromq.com, feedback would be welcome
[Wednesday 01 September 2010] [13:55:45] <pieterh> this is meant to be the welcome site for new 0MQ users
[Wednesday 01 September 2010] [13:56:10] <pieterh> next I’ll remove a lot of the fluff from the zeromq.org front page and menus
[Wednesday 01 September 2010] [14:00:50] <pieterh> my idea is to make the zeromq.org site an index of all projects
[Wednesday 01 September 2010] [14:01:00] <pieterh> kind of ranked by activity somehow
[Wednesday 01 September 2010] [14:05:50] <pieterh> no-one minds? :-)
[Wednesday 01 September 2010] [14:09:03] Quit pieterh has left this server (Read error: Operation timed out).
[Wednesday 01 September 2010] [14:12:16] <Steve-o> inproc is still a new word
[Wednesday 01 September 2010] [14:13:32] <guido_g> the onhover bold is a little overdone
[Wednesday 01 September 2010] [14:15:46] Quit alfborge has left this server (Quit: leaving).
[Wednesday 01 September 2010] [14:16:00] <guido_g> on the page linked from 04READ should show the canonical doku first (the guide & the manual)
[Wednesday 01 September 2010] [14:16:56] <jond> steve-o: agree about inproc, not widely used AFAIK
[Wednesday 01 September 2010] [14:23:39] Quit pieterh has left this server (Ping timeout: 258 seconds).
[Wednesday 01 September 2010] [15:45:07] <mcxx> I got freshly cloned zeromq2 and pyzmq repos from github
[Wednesday 01 September 2010] [16:00:54] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [16:27:03] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 01 September 2010] [16:46:14] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Wednesday 01 September 2010] [17:27:20] Quit jond has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [17:57:57] Quit jsimmons has left this server (Client Quit).
[Wednesday 01 September 2010] [18:27:24] Quit AndrewBC has left this server (Ping timeout: 260 seconds).
[Wednesday 01 September 2010] [18:38:18] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [18:42:55] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Wednesday 01 September 2010] [19:05:24] Quit plq has left this server (Ping timeout: 258 seconds).
[Wednesday 01 September 2010] [19:18:54] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [19:30:09] Quit plq has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [20:03:27] <ModusPwnens> Hi, I’m having an issue with the first byte of a message being corrupted when it is sent. Has anyone experienced this?
[Wednesday 01 September 2010] [20:05:43] <bgranger> mcxx: You need to grab the stable 2.0.7 releases of pyzmq and zeromq.
[Wednesday 01 September 2010] [20:06:02] <bgranger> Trunk of pyzmq works with stable 2.0.8 of zeromq.
[Wednesday 01 September 2010] [20:06:15] <bgranger> We have not started following the post 2.0.8 master of zeromq
[Wednesday 01 September 2010] [20:06:44] <ModusPwnens> clarification, it’s actually the first two bytes that get corrupted
[Wednesday 01 September 2010] [20:07:40] Quit MJW has left this server (Client Quit).
[Wednesday 01 September 2010] [20:08:42] Quit ModusPwnens has left this server.
[Wednesday 01 September 2010] [20:47:00] Quit
mw__ has left this server (Quit: Alea iactus est.).
[Wednesday 01 September 2010] [22:08:48] Quit miguelito has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [22:40:34] <lestrrat> I assume this is the right place to talk about zeromq?
[Wednesday 01 September 2010] [22:42:31] <lestrrat> I just wanted to make if zeromq’s recv() is supposed to bail out properly (or not) when it receives a signal while it’s waiting for a message?
[Wednesday 01 September 2010] [22:42:49] <lestrrat> my perl binding seems to be stuck, and I’m trying to figure out if it’s my problem or not.
[Wednesday 01 September 2010] [23:46:46] <bgranger> lestrrat: I think I know what is going on.
[Wednesday 01 September 2010] [23:46:58] <bgranger> Are you still around?
[Wednesday 01 September 2010] [23:47:02] <lestrrat> yes!
[Wednesday 01 September 2010] [23:47:13] <bgranger> Hey, we have the same issue in the Python bindings.
[Wednesday 01 September 2010] [23:47:27] <bgranger> It perl is like Python, then it sets its own signal handlers.
[Wednesday 01 September 2010] [23:47:33] <lestrrat> ah, so glad I’m not alone on this :)
[Wednesday 01 September 2010] [23:47:34] <lestrrat> right
[Wednesday 01 September 2010] [23:47:47] <bgranger> BUT, while a blocking recv is waiting, perl is completely out of control.
[Wednesday 01 September 2010] [23:48:22] <AndrewBC> I wrapped mine in a try except in python, excepting a KeyboardInterrupt
[Wednesday 01 September 2010] [23:48:25] <AndrewBC> worked fine
[Wednesday 01 September 2010] [23:48:42] <bgranger> SO what happens is that Perl’s signal handler will receive the signal, but it won’t be handled until the recv returns. To test if this is what is really going on, try to create a simple example , then send the signal and send an additional msg to the socket so that recv returns.
[Wednesday 01 September 2010] [23:48:44] <AndrewBC> so you might have to manually catch ^C if that’s possible in perl
[Wednesday 01 September 2010] [23:49:14] <bgranger> AndrewBC, it won’t work if there is not a msg recv’d
[Wednesday 01 September 2010] [23:49:28] <bgranger> Or I should say, the KeyboardInt won’t be raised until recv returns.
[Wednesday 01 September 2010] [23:49:37] <lestrrat> bgranger: ah right
[Wednesday 01 September 2010] [23:49:37] <AndrewBC> oh, hm
[Wednesday 01 September 2010] [23:49:38] <bgranger> You can’t really interupt it with SIGINT
[Wednesday 01 September 2010] [23:50:22] <lestrrat> I can go and confirm, but if that’s the case, it’s really zeromq’s responsibility to return on a signal, no? (with errno = EINTER or whatever)
[Wednesday 01 September 2010] [23:50:35] <bgranger> But the same is true when Python calls any external C/C++ code.
[Wednesday 01 September 2010] [23:50:40] <lestrrat> right
[Wednesday 01 September 2010] [23:50:55] <bgranger> No, because Python’s signal handler is still in place, it just doesn’t get to run.
[Wednesday 01 September 2010] [23:51:20] <lestrrat> yeah, but we can propagate the signal as long as we get back control
[Wednesday 01 September 2010] [23:51:58] <bgranger> But you can’t get back control
[Wednesday 01 September 2010] [23:52:02] <lestrrat> hmm?
[Wednesday 01 September 2010] [23:52:26] <bgranger> The signal handler is the one at the C level, not the one that you can control using the signal module
[Wednesday 01 September 2010] [23:52:41] <lestrrat> yeah, but I’m writing the Perl<->C binding, so I have control :)
[Wednesday 01 September 2010] [23:53:01] <bgranger> To change the behavior you actually to have use signal.h and set jump/long jump logic.
[Wednesday 01 September 2010] [23:53:07] <bgranger> But it can be done
[Wednesday 01 September 2010] [23:54:00] <bgranger> It is just subtle logic and hard to get right in a cross platform manner.
[Wednesday 01 September 2010] [23:54:15] <bgranger> I don’t think any of the language bindings are doing this yet, but eventually all should
[Wednesday 01 September 2010] [23:54:46] <lestrrat> I dunno the python internals, but I don’t think we need a jump logic for perl
[Wednesday 01 September 2010] [23:54:49] <bgranger> Meanwhile, we are just not using blocking recv when it needs to be interupted
[Wednesday 01 September 2010] [23:55:32] <bgranger> If perl sets its own signal handlers at the C level, you likely will.
[Wednesday 01 September 2010] [23:55:35] <lestrrat> perl’s signal handling doesn’t run on “real time”. it’s deferred until the next perl op. so as long as I’m in C land, I can tell the Perl interpreter that there was a signal, and properly emulate it.
[Wednesday 01 September 2010] [23:56:33] <bgranger> That is how Python works as well and it why the default signal handler doesn’t actually interrupt “in real time:”
[Wednesday 01 September 2010] [23:56:58] <lestrrat> then I don’t see why you need a jump...? what am I missing?
[Wednesday 01 September 2010] [23:57:46] <bgranger> I don’t remember the details honestly. I think the issue is that you have to be very careful about restoring Python’s regular signal handler and putting the intepreter back in the right place in the stack.
[Wednesday 01 September 2010] [23:57:57] <lestrrat> hmm.
[Wednesday 01 September 2010] [23:58:25] <bgranger> That is, if you want control to return to Perl/Python in a way that doesn’t simply kill the process.
[Wednesday 01 September 2010] [23:58:38] <lestrrat> actually, I don’t really care either way – I don’t have to have zmq_recv to get back to the Perl handler – I just need it to get back to me once a signal is sent w/o killing the process
[Wednesday 01 September 2010] [23:58:45] <lestrrat> otherwise servers hang.
[Wednesday 01 September 2010] [23:59:01] <bgranger> yep
[Wednesday 01 September 2010] [23:59:14] <lestrrat> glad to know I’m the only one. I can safely write an RFE :)
[Thursday 02 September 2010] [00:43:40] <sustrik> bgranger, lestrrat: i am still not sure about EINTR behaviour
[Thursday 02 September 2010] [00:43:53] <sustrik> would returning EINTR from blocking call help in any way?
[Thursday 02 September 2010] [00:47:53] <bgranger> In what situations?
[Thursday 02 September 2010] [00:49:02] <bgranger> Are you thinking of installing a signal handler that catches SIGINT and translates that into EINTR?
[Thursday 02 September 2010] [00:49:33] <bgranger> That would help us, but that signal handler would have to play nice with that of Python
[Thursday 02 September 2010] [00:52:11] <sustrik> bgragner: no, i meant the peculiar functionality of 0mq
[Thursday 02 September 2010] [00:52:25] <sustrik> that catches OS’s EINTR
[Thursday 02 September 2010] [00:52:34] <sustrik> and instead of returning it to the user
[Thursday 02 September 2010] [00:52:45] <sustrik> just restarts the blocking operation
[Thursday 02 September 2010] [00:53:01] <bgranger> I am not familiar with what zmq is doing right now
[Thursday 02 September 2010] [00:53:07] <sustrik> but afaiu that’s not the problem you are dealing with
[Thursday 02 September 2010] [00:53:30] <bgranger> When does the OS signal EINTR?
[Thursday 02 September 2010] [00:53:47] <bgranger> And how does zeromq currently handle it?
[Thursday 02 September 2010] [00:54:00] <sustrik> if a threads gets a signal while in blocking operation
[Thursday 02 September 2010] [00:54:33] <sustrik> 0mq currently ignores it, i.e. restarts the blocking op
[Thursday 02 September 2010] [00:56:29] <bgranger> I guess what I don’t know if how all of this interplays with Python’s signal handling.
[Thursday 02 September 2010] [00:56:35] <sustrik> exactly
[Thursday 02 September 2010] [00:56:43] <bgranger> Python only receives signals in the main thread
[Thursday 02 September 2010] [00:56:55] <bgranger> Does zeromq install any signal handlers?
[Thursday 02 September 2010] [00:56:57] <sustrik> that’s the one that’s blocked in 0mq?
[Thursday 02 September 2010] [00:57:05] <sustrik> no, it does not
[Thursday 02 September 2010] [00:57:10] <bgranger> OK
[Thursday 02 September 2010] [00:57:21] <bgranger> but when a socket call returns EINTR, it just continues
[Thursday 02 September 2010] [00:57:28] <sustrik> right
[Thursday 02 September 2010] [00:57:34] <sustrik> actaully, you can check what’s happening yourself
[Thursday 02 September 2010] [00:57:37] <bgranger> OK, then another ?
[Thursday 02 September 2010] [00:57:41] <bgranger> OK
[Thursday 02 September 2010] [00:57:46] <bgranger> How...
[Thursday 02 September 2010] [00:57:46] <sustrik> when waiting for a message
[Thursday 02 September 2010] [00:57:59] <sustrik> 0mq is stuck in a read call
[Thursday 02 September 2010] [00:58:06] <bgranger> Correct
[Thursday 02 September 2010] [00:58:09] <sustrik> let me find the exact code...
[Thursday 02 September 2010] [00:58:18] <bgranger> OK great, that would be helpful.
[Thursday 02 September 2010] [00:59:27] <bgranger> Are the other occasions where EINTR is returned?
[Thursday 02 September 2010] [01:00:06] <sustrik> polling
[Thursday 02 September 2010] [01:00:12] <sustrik> are you on linux?
[Thursday 02 September 2010] [01:00:16] <lestrrat> sorry was away
[Thursday 02 September 2010] [01:00:21] <bgranger> linux and Mac
[Thursday 02 September 2010] [01:00:22] <sustrik> hi
[Thursday 02 September 2010] [01:00:31] <bgranger> Talking about signal handling more...
[Thursday 02 September 2010] [01:00:37] <sustrik> trunk or 2.0.8?
[Thursday 02 September 2010] [01:00:55] <bgranger> pyzmq doesn’t work with trunk yet, sso 2.0.8 or 2.0.7
[Thursday 02 September 2010] [01:00:58] <lestrrat> I’m using 2.0.8
[Thursday 02 September 2010] [01:01:53] <bgranger> but if we can do signal -> EINTR -> Python bindings -> raise KeyboardInterrupt that would be great
[Thursday 02 September 2010] [01:02:04] <sustrik> yes
[Thursday 02 September 2010] [01:02:14] <sustrik> you see the loop, right?
[Thursday 02 September 2010] [01:02:21] <lestrrat> right
[Thursday 02 September 2010] [01:02:28] <bgranger> Yep
[Thursday 02 September 2010] [01:02:33] <sustrik> just put a printf there or something
[Thursday 02 September 2010] [01:02:42] <sustrik> and check whether it loops in case of SIGINT
[Thursday 02 September 2010] [01:02:48] <bgranger> OK I will do that
[Thursday 02 September 2010] [01:02:56] <bgranger> I will check on Linux and OS X
[Thursday 02 September 2010] [01:02:57] <sustrik> if so we can return EINTR instead of looping
[Thursday 02 September 2010] [01:03:00] <sustrik> great
[Thursday 02 September 2010] [01:03:01] <sustrik> thanks
[Thursday 02 September 2010] [01:03:04] <bgranger> Cool
[Thursday 02 September 2010] [01:03:07] <bgranger> Thanks!
[Thursday 02 September 2010] [01:03:15] <bgranger> Won’t get to it tonight though
[Thursday 02 September 2010] [01:03:21] <bgranger> Getting late here...
[Thursday 02 September 2010] [01:03:39] <sustrik> sure
[Thursday 02 September 2010] [01:03:53] <sustrik> lestrrat: you may try the same, if in hurry
[Thursday 02 September 2010] [01:03:55] <lestrrat> yeah, I’m on $day_job too, so I’m not going to be able to check it right away. Will see if I can sneak that in in the next couple of hours
[Thursday 02 September 2010] [01:04:01] <bgranger> If I want to try the change, I can just have the function return EINTR?
[Thursday 02 September 2010] [01:04:11] <lestrrat> yep, will do when pointy haired boss isn’t looking
[Thursday 02 September 2010] [01:04:17] <sustrik> :)
[Thursday 02 September 2010] [01:04:18] <lestrrat> AFK for now
[Thursday 02 September 2010] [01:04:28] <bgranger> Will the return value of that function be returned to the caller
[Thursday 02 September 2010] [01:04:33] <sustrik> bgranger: probably
[Thursday 02 September 2010] [01:04:46] <sustrik> let me check
[Thursday 02 September 2010] [01:04:52] <bgranger> Well, it looks like it must return a bool though
[Thursday 02 September 2010] [01:05:07] <bgranger> May have to change more logic.
[Thursday 02 September 2010] [01:05:16] <sustrik> ah, ok
[Thursday 02 September 2010] [01:05:24] <sustrik> it’s not propagated
[Thursday 02 September 2010] [01:05:30] <sustrik> but i can fix that
[Thursday 02 September 2010] [01:05:36] <sustrik> the question is whether it would help
[Thursday 02 September 2010] [01:05:38] <bgranger> OK, let’s test first
[Thursday 02 September 2010] [01:05:40] <bgranger> right
[Thursday 02 September 2010] [01:09:00] <sustrik> lestrrat: are you in japan? i’ve notices some of your twitter notes
[Thursday 02 September 2010] [01:09:12] <sustrik> had no idea what they were about though :)
[Thursday 02 September 2010] [01:10:29] <guido_g> hi all
[Thursday 02 September 2010] [01:10:37] <guido_g> good morning sustrik
[Thursday 02 September 2010] [01:10:44] <sustrik> morning
[Thursday 02 September 2010] [01:11:11] <guido_g> did you read that the poll segfault in master is still there?
[Thursday 02 September 2010] [01:11:38] <sustrik> yep, seen you saying it yesterday
[Thursday 02 September 2010] [01:11:42] <sustrik> can i reproduce it?
[Thursday 02 September 2010] [01:11:50] <guido_g> i hope so
[Thursday 02 September 2010] [01:11:58] <sustrik> which test program?
[Thursday 02 September 2010] [01:12:28] <guido_g> zmqcpp and sender
[Thursday 02 September 2010] [01:12:42] <guido_g> zmqcpp segfaults in poll
[Thursday 02 September 2010] [01:13:01] <sustrik> what order to start them in?
[Thursday 02 September 2010] [01:13:20] <guido_g> i started zmqcpp first and then sender
[Thursday 02 September 2010] [01:13:26] <guido_g> sender is a one shot thing
[Thursday 02 September 2010] [01:13:30] <sustrik> ok
[Thursday 02 September 2010] [01:14:18] <guido_g> going to write a mq test program for that
[Thursday 02 September 2010] [01:14:24] <sustrik> ack
[Thursday 02 September 2010] [01:14:44] <guido_g> seems to be a nasty corner that needs some regression
[Thursday 02 September 2010] [01:22:52] <sustrik> guido_g:
[Thursday 02 September 2010] [01:22:54] <sustrik> In file included from zmqcpp.cpp:9:
[Thursday 02 September 2010] [01:22:54] <sustrik> worker.hpp:14:21: error: pqxx/pqxx: No such file or directory
[Thursday 02 September 2010] [01:22:56] <sustrik> ...
[Thursday 02 September 2010] [01:23:05] <guido_g> oh shit
[Thursday 02 September 2010] [01:23:07] <guido_g> sorry
[Thursday 02 September 2010] [01:23:14] <sustrik> I’ll rather wait for your test program
[Thursday 02 September 2010] [01:23:21] <guido_g> ack
[Thursday 02 September 2010] [01:23:29] <guido_g> never mind then
[Thursday 02 September 2010] [01:23:45] <guido_g> “i’ll be back!” :)
[Thursday 02 September 2010] [01:24:07] <sustrik> ;)
[Thursday 02 September 2010] [01:24:45] <guido_g> so, day job is waiting, cu
[Thursday 02 September 2010] [01:24:57] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [01:31:01] <lestrrat> sustrik: yes I’m in Japan :)
[Thursday 02 September 2010] [01:32:01] <sustrik> lot of japanese tweets about 0mq lately
[Thursday 02 September 2010] [01:32:04] <sustrik> it’s fun
[Thursday 02 September 2010] [01:33:44] <lestrrat> let’s see if I can work on that signal thing...
[Thursday 02 September 2010] [01:34:11] <sustrik> ok
[Thursday 02 September 2010] [01:40:09] <lestrrat> alright, so I’m a autoconf noob. how do I make this go away?:
[Thursday 02 September 2010] [01:40:16] Notice -lestrrat to #zeromq- ./configure: line 18331: syntax error near unexpected token
`GLIB,’
[Thursday 02 September 2010] [01:40:17] Notice -lestrrat to #zeromq- ./configure: line 18331: ` PKG_CHECK_MODULES(GLIB, glib-2.0 gthread-2.0)’
[Thursday 02 September 2010] [01:41:27] <sustrik> you need newer version of autoconf
[Thursday 02 September 2010] [01:41:34] <sustrik> are you on OSX?
[Thursday 02 September 2010] [01:42:40] <sustrik> this one would deserve going into FAQ btw
[Thursday 02 September 2010] [01:43:01] <lestrrat> yeah, I’m on OSX
[Thursday 02 September 2010] [01:43:18] <sustrik> it’s a known issue
[Thursday 02 September 2010] [01:43:22] <lestrrat> grrr
[Thursday 02 September 2010] [01:43:30] * lestrrat looks for autoconf in homebrew
[Thursday 02 September 2010] [01:43:34] <sustrik> osx has old version of autoconf installed by default
[Thursday 02 September 2010] [01:44:26] <lestrrat> grr, and it’s not there in homebrew
[Thursday 02 September 2010] [01:47:35] <lestrrat> man, this is going to take some time :)
[Thursday 02 September 2010] [01:47:37] <sustrik> lestrrat: sorry, it’s pkg-config
[Thursday 02 September 2010] [01:47:55] * sustrik is autotools noob as well
[Thursday 02 September 2010] [01:47:58] <lestrrat> oh cool
[Thursday 02 September 2010] [01:48:52] <lestrrat> hmm, I seem to have the latest pkg-config
[Thursday 02 September 2010] [01:49:25] <sustrik> are you building witg –with-pgm btw?
[Thursday 02 September 2010] [01:50:15] <lestrrat> nay, just the default with a custom –prefix
[Thursday 02 September 2010] [01:50:19] <sustrik> hm
[Thursday 02 September 2010] [01:50:28] <sustrik> well, you’ll have to ask on the mailing list
[Thursday 02 September 2010] [01:50:38] <sustrik> as i say, i’m an autotools idiot
[Thursday 02 September 2010] [01:53:06] <lestrrat> trying out autoconf anyways
[Thursday 02 September 2010] [02:09:07] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rd5b6f68 10/ AUTHORS : Mikael Kjaer added to AUTHORS -
http://bit.ly/aI6jV2[Thursday 02 September 2010] [02:09:08] <CIA-20> zeromq2: 03Bernd Melchers 07maint * r8ec0743 10/ (AUTHORS src/signaler.cpp): Fix for signaler_t on HP-UX and AIX platforms -
http://bit.ly/9XM0R0[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: 03Jon Dyte 07maint * r14853c2 10/ (src/prefix_tree.cpp src/prefix_tree.hpp):
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: This is because as it adds the 255th element at a node it attempts to calculate
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: the count member var which is an unsigned char via count = (255 -0) + 1; and
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: pass the result to realloc. Unfortunately the result is zero and realloc returns
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: null; the prefix_tree asserts. I have fixed it by making the count an unsigned
[Thursday 02 September 2010] [02:09:35] <CIA-20> zeromq2: 03Martin Sustrik 07master * rd5b6f68 10/ AUTHORS : Mikael Kjaer added to AUTHORS -
http://bit.ly/aI6jV2[Thursday 02 September 2010] [02:09:36] <CIA-20> zeromq2: 03Bernd Melchers 07master * r8ec0743 10/ (AUTHORS src/signaler.cpp): Fix for signaler_t on HP-UX and AIX platforms -
http://bit.ly/9XM0R0[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: 03Jon Dyte 07master * r14853c2 10/ (src/prefix_tree.cpp src/prefix_tree.hpp):
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: This is because as it adds the 255th element at a node it attempts to calculate
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: the count member var which is an unsigned char via count = (255 -0) + 1; and
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: pass the result to realloc. Unfortunately the result is zero and realloc returns
[Thursday 02 September 2010] [02:09:40] <CIA-20> zeromq2: null; the prefix_tree asserts. I have fixed it by making the count an unsigned
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0a1f7e3 10/ (AUTHORS src/signaler.cpp src/trie.cpp src/trie.hpp): (log message trimmed)
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Merge branch ‘maint’
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: * maint:
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Fix for signaler_t on HP-UX and AIX platforms
[Thursday 02 September 2010] [02:10:04] <CIA-20> zeromq2: Mikael Kjaer added to AUTHORS
[Thursday 02 September 2010] [02:10:04] <CIA-20> zeromq2: Conflicts:
[Thursday 02 September 2010] [02:16:49] Quit bgranger has left this server (Quit: bgranger).
[Thursday 02 September 2010] [02:35:56] <lestrrat> hmm, zeromq HEAD against my perl binding doesn’t work – (seeing some tests block on odd places)
[Thursday 02 September 2010] [02:36:02] <lestrrat> could be something on my part, of course.
[Thursday 02 September 2010] [02:36:05] <lestrrat> investigating...
[Thursday 02 September 2010] [02:52:46] <lestrrat> hmm, switching back to 2.0.8 makes the problem go away. I see all my destructors running, so I’m willing to be there’s some sort of background thread or something that’s kept alive in zeromq
[Thursday 02 September 2010] [02:54:03] <guido_g> re
[Thursday 02 September 2010] [02:58:55] <sustrik> lestrrat: HEAD is a dev version, testing with 2.0.8 is ok
[Thursday 02 September 2010] [02:59:03] <lestrrat> k
[Thursday 02 September 2010] [02:59:12] <sustrik> what about the SIGINT thing?
[Thursday 02 September 2010] [02:59:49] <lestrrat> trying now
[Thursday 02 September 2010] [03:02:10] <lestrrat> so yeah, looks like it’s stuck in that loop
[Thursday 02 September 2010] [03:02:29] <lestrrat> tweaking to see if I can cleanly exit there...
[Thursday 02 September 2010] [03:05:37] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [03:11:16] <sustrik> lestrrat: i’m going to be away now for a while, but leave me a message here about whether returning EINTR from 0mq would help you
[Thursday 02 September 2010] [03:11:37] <lestrrat> yep, thanks
[Thursday 02 September 2010] [03:23:34] <lestrrat> this works for me!
[Thursday 02 September 2010] [03:41:43] Quit grahamalot has left this server (Quit: grahamalot).
[Thursday 02 September 2010] [04:30:52] Quit pieterh has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [04:45:31] <sustrik> lestrrat: great
[Thursday 02 September 2010] [04:45:51] <sustrik> i’ll fix it in a bit different way (actually returning EINTR)
[Thursday 02 September 2010] [04:46:04] <sustrik> but it’s good news to know it solves the Ctrl+C problem
[Thursday 02 September 2010] [04:46:40] <sustrik> we’ll see whether it helps with python as well
[Thursday 02 September 2010] [04:52:42] Quit gebi has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [04:58:39] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 02 September 2010] [05:00:31] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 02 September 2010] [05:19:30] Quit AndrewBC has left this server (Ping timeout: 258 seconds).
[Thursday 02 September 2010] [05:21:08] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [05:23:35] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [05:23:42] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 02 September 2010] [05:52:32] <mato> sustrik: are you there?
[Thursday 02 September 2010] [06:00:55] <sssss> test
[Thursday 02 September 2010] [06:01:15] Quit sssss has left this server (Client Quit).
[Thursday 02 September 2010] [06:06:57] <sustrik> mato: re
[Thursday 02 September 2010] [06:14:51] <mato> sustrik: could you switch on all the test servers + switch please?
[Thursday 02 September 2010] [06:15:15] <mato> sustrik: actually, just one of the 8-core boxes is all i need right now
[Thursday 02 September 2010] [06:15:32] <mato> sustrik: and the two white ones
[Thursday 02 September 2010] [06:15:51] <mato> sustrik: so pick the quieter of janos or csaba :) i don’t care...
[Thursday 02 September 2010] [06:16:23] <sustrik> oh my
[Thursday 02 September 2010] [06:16:24] <sustrik> ok
[Thursday 02 September 2010] [06:16:41] * sustrik is inserting earplugs
[Thursday 02 September 2010] [06:16:46] <mato> :)
[Thursday 02 September 2010] [06:16:59] <mato> loud music also works :)
[Thursday 02 September 2010] [06:18:13] <sustrik> started
[Thursday 02 September 2010] [06:18:30] <mato> damn
[Thursday 02 September 2010] [06:18:31] <sustrik> now i realise there’s just one of those big boxes here
[Thursday 02 September 2010] [06:18:38] <sustrik> where’s the other one?
[Thursday 02 September 2010] [06:18:39] <pieterh> hey, the lights just dimmed here...!
[Thursday 02 September 2010] [06:18:50] <sustrik> still at malosek’s place?
[Thursday 02 September 2010] [06:18:52] <mato> sustrik: just one? i dunno? does malo still have the other?
[Thursday 02 September 2010] [06:19:02] <mato> sustrik: i can’t connect to your IP at all :(
[Thursday 02 September 2010] [06:19:22] <mato> hmm
[Thursday 02 September 2010] [06:19:29] <sustrik> i see the world outside
[Thursday 02 September 2010] [06:19:55] <mato> i have a suspicion your chello connection is funny
[Thursday 02 September 2010] [06:20:08] <sustrik> ?
[Thursday 02 September 2010] [06:20:22] <mato> well, first of all, your ip is on some weird subnet
[Thursday 02 September 2010] [06:20:42] <mato> which i’ve never seen
[Thursday 02 September 2010] [06:20:49] <mato> 2nd, i can’t ping or connect to it :(
[Thursday 02 September 2010] [06:21:11] <sustrik> :(
[Thursday 02 September 2010] [06:21:31] <sustrik> you can take the boxes
[Thursday 02 September 2010] [06:34:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [06:44:06] <guido_g> openvpn is really nice for this kind of things
[Thursday 02 September 2010] [06:45:44] <sustrik> it works now (not sure why)
[Thursday 02 September 2010] [07:10:56] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 02 September 2010] [07:24:02] Quit pieterh has left this server (Ping timeout: 272 seconds).
[Thursday 02 September 2010] [07:26:13] <mato> wow
[Thursday 02 September 2010] [07:26:19] <mato> sustrik: quick question...
[Thursday 02 September 2010] [07:26:40] <mato> sustrik: zmq_free_fn() is called in the context of the application or i/o thread?
[Thursday 02 September 2010] [07:27:24] <sustrik> i/o
[Thursday 02 September 2010] [07:27:28] <mato> riiiiiight
[Thursday 02 September 2010] [07:27:33] <mato> that explains a lot
[Thursday 02 September 2010] [07:27:48] <sustrik> ?
[Thursday 02 September 2010] [07:28:05] <mato> i’m having issues with the python GIL
[Thursday 02 September 2010] [07:28:18] <mato> at least i think that’s what happening
[Thursday 02 September 2010] [07:28:24] <sustrik> a-ha, i see
[Thursday 02 September 2010] [07:28:31] <mato> it explains all the mysterious hanging close() with 2.0.x
[Thursday 02 September 2010] [07:28:38] <sustrik> python zero-copy functionality
[Thursday 02 September 2010] [07:28:43] <mato> ja
[Thursday 02 September 2010] [07:28:59] <mato> anyway, i’m going to lunch, we can discuss when i get back...
[Thursday 02 September 2010] [07:29:03] <sustrik> ok
[Thursday 02 September 2010] [07:38:29] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Thursday 02 September 2010] [07:47:22] <sustrik> thanks
[Thursday 02 September 2010] [07:47:37] <sustrik> have you thought of a pub to meet the other guys in?
[Thursday 02 September 2010] [07:47:50] <sustrik> it’s only 3 people so far
[Thursday 02 September 2010] [07:47:58] <sustrik> so no need for complex arrangements
[Thursday 02 September 2010] [07:48:18] <sustrik> say it’ll be 5-7 ultimately
[Thursday 02 September 2010] [07:48:49] <sustrik> oops :)
[Thursday 02 September 2010] [07:48:57] <sustrik> that was intended for another window
[Thursday 02 September 2010] [07:49:03] <sustrik> the london meetup stuff
[Thursday 02 September 2010] [08:16:57] <mikko> sustrik: theodore bullfrog is a decent place
[Thursday 02 September 2010] [08:18:58] <sustrik> looks central enough
[Thursday 02 September 2010] [08:20:46] <sustrik> is that restaurant or a pub?
[Thursday 02 September 2010] [08:24:12] <sustrik> mikko: hullo!
[Thursday 02 September 2010] [08:24:53] <mikko> pub
[Thursday 02 September 2010] [08:24:57] <mikko> but they serve food as well
[Thursday 02 September 2010] [08:25:27] <sustrik> ok, why not then
[Thursday 02 September 2010] [08:25:47] Quit guido_g has left this server (Disconnected by services).
[Thursday 02 September 2010] [08:25:59] Nick OldNugget is now known as guido_g.
[Thursday 02 September 2010] [08:26:31] <sustrik> nice
[Thursday 02 September 2010] [08:36:17] <mato> sustrik: ok, so this GIL thing
[Thursday 02 September 2010] [08:36:24] <mato> sustrik: it’s an interesting problem
[Thursday 02 September 2010] [08:37:13] <sustrik> yes?
[Thursday 02 September 2010] [08:37:24] <mato> what i see is that in my python application zmq_close() is getting called, due to Python gc kicking in
[Thursday 02 September 2010] [08:37:57] <mato> however, at the same time, zmq_msg_close() on a method shared with Python is called in the i/o thread
[Thursday 02 September 2010] [08:38:35] <mato> now, zmq_close() calls process_commands() which is waiting for a signal from the i/o thread
[Thursday 02 September 2010] [08:39:00] <mato> but that signal never arrives, because the free_fn() called by the zmq_msg_close() in the i/o thread is trying to acquire the Python GIL
[Thursday 02 September 2010] [08:39:05] <mato> -> deadlock
[Thursday 02 September 2010] [08:39:47] <mato> you follow me?
[Thursday 02 September 2010] [08:40:11] <mato> anyway, i can work around it now by not closing yet more sockets :)
[Thursday 02 September 2010] [08:40:17] <mato> but it’s an interesting case
[Thursday 02 September 2010] [08:40:35] <mato> i will have to make a test case and write it up on the mailing list so that other people can participate
[Thursday 02 September 2010] [08:40:44] <mato> it’s not clear what the solution is
[Thursday 02 September 2010] [08:40:48] <sustrik> i have to draw it down :)
[Thursday 02 September 2010] [08:41:02] <sustrik> mato: yes, please
[Thursday 02 September 2010] [08:41:06] <sustrik> so that it’s not lost
[Thursday 02 September 2010] [08:41:09] <mato> definitely
[Thursday 02 September 2010] [08:41:18] <mato> it needs to be solved, and i suspect the same problem can occur in 2.1
[Thursday 02 September 2010] [08:41:34] <mato> however, it’s possible the solution needs to be done on the pyzmq side
[Thursday 02 September 2010] [08:43:16] <lestrrat> sustrik: wrt EINTR thing, what version should I be expecting it to be implemented? just want to know, cause I’d like to wait my first release of mongrel2 handler until that’s fixed.
[Thursday 02 September 2010] [08:43:35] <lestrrat> don’t want to tell my users to send a SIGKILL to stop their handlers :)
[Thursday 02 September 2010] [08:43:53] <lestrrat> (not in a hurry, just want to get an idea)
[Thursday 02 September 2010] [08:44:02] <sustrik> lestrrat: the problem is that it isn’t backward compatible
[Thursday 02 September 2010] [08:44:07] <sustrik> :|
[Thursday 02 September 2010] [08:44:08] <lestrrat> ah
[Thursday 02 September 2010] [08:44:12] <mato> EINTR again ?
[Thursday 02 September 2010] [08:44:12] <lestrrat> yeah, true
[Thursday 02 September 2010] [08:44:15] <sustrik> yes
[Thursday 02 September 2010] [08:44:34] <sustrik> it turns out that Ctrl+C problem in bindings
[Thursday 02 September 2010] [08:44:41] <sustrik> can be solved by returning EINTR
[Thursday 02 September 2010] [08:44:50] <mato> sustrik: have you verified this?
[Thursday 02 September 2010] [08:45:02] <sustrik> lestrrat did with perl binding
[Thursday 02 September 2010] [08:45:15] <mato> hmm, i think i know why it “solves” the problem
[Thursday 02 September 2010] [08:45:21] <mato> or i can guess...
[Thursday 02 September 2010] [08:45:28] <sustrik> i’m still waiting for brian to verify it with python
[Thursday 02 September 2010] [08:45:29] <lestrrat> I’m willing to accept I’m a bit off :)
[Thursday 02 September 2010] [08:45:48] <sustrik> mato: go on
[Thursday 02 September 2010] [08:46:01] <mato> i reckon what happens is returning EINTR means “the language runtime” wakes up
[Thursday 02 September 2010] [08:46:18] <sustrik> right
[Thursday 02 September 2010] [08:46:35] <mato> depending on said language runtime’s signal handling (out of our control, obviously), the runtime waking up may mean that it processes pending signals
[Thursday 02 September 2010] [08:46:50] <mato> hence the “solution”
[Thursday 02 September 2010] [08:47:06] <sustrik> well, it least the runtime has a chance to process the signal
[Thursday 02 September 2010] [08:47:16] <mato> hmm
[Thursday 02 September 2010] [08:47:20] <sustrik> right now it’s completely stuck
[Thursday 02 September 2010] [08:47:23] <mato> yeah
[Thursday 02 September 2010] [08:47:43] <mato> of course the runtime might be doing something entirely different
[Thursday 02 September 2010] [08:47:50] <mato> so it’s not a guaranteed solution
[Thursday 02 September 2010] [08:47:53] <sustrik> not our fault
[Thursday 02 September 2010] [08:47:56] <mato> just one that might help some people
[Thursday 02 September 2010] [08:48:16] <mato> hmm, except...
[Thursday 02 September 2010] [08:48:19] <sustrik> obviously we cannot give guarantees for broken runtimes
[Thursday 02 September 2010] [08:48:31] <mato> yes, but what is your proposed implementation?
[Thursday 02 September 2010] [08:48:44] <sustrik> forward EINTR to the user
[Thursday 02 September 2010] [08:48:48] <mato> yes, except...
[Thursday 02 September 2010] [08:49:08] <mato> the relationship between EINTR from a syscall and SIGINT is afaik nondeterministic
[Thursday 02 September 2010] [08:49:17] <mato> in other words, you might get an EINTR back from a blocking call
[Thursday 02 September 2010] [08:49:18] <mato> or not
[Thursday 02 September 2010] [08:49:26] <mato> you see where this is heading?
[Thursday 02 September 2010] [08:49:31] <lestrrat> should I explain how perl’s signal handling works? if you guys know what’s up, I’m not an entirely an expert either so I’ll let you guys debate the actual implementation ;)
[Thursday 02 September 2010] [08:49:43] <mato> lestrrat: sure, go for it?
[Thursday 02 September 2010] [08:49:45] <mato> .
[Thursday 02 September 2010] [08:50:56] <lestrrat> well, the important part wrt to all of this is that, when we go down to the C world, Perl’s signal handling is effectively silenced.
[Thursday 02 September 2010] [08:51:12] <lestrrat> (C world, as in binding)
[Thursday 02 September 2010] [08:51:42] <lestrrat> when the binding method is called, things are completely left up to the C/C++ routine, so you can do whatever with the signals
[Thursday 02 September 2010] [08:51:59] <lestrrat> and when zmq’s recv() is being called, perl has absolutely no say there
[Thursday 02 September 2010] [08:52:33] <lestrrat> so from perl’s perspective, all it needs is for recv() to return, and let it know from a return value or errno that there was a signal sent somewhere along the line
[Thursday 02 September 2010] [08:53:06] <lestrrat> then (if need be) the Perl binding can tell the perl interpreter that there was a signal, and let perl’s sig handlers run
[Thursday 02 September 2010] [08:53:24] <lestrrat> at least that’s how I understand it to work.
[Thursday 02 September 2010] [08:53:42] <sustrik> right
[Thursday 02 September 2010] [08:54:03] <mato> ok
[Thursday 02 September 2010] [08:54:18] <mato> i’ve not looked into the perl signal handling
[Thursday 02 September 2010] [08:54:18] <sustrik> what i think mato is saying is that OS does not have to interrupt a blocking call (at C level) when SIGINT happens
[Thursday 02 September 2010] [08:54:24] <mato> yes
[Thursday 02 September 2010] [08:54:32] <lestrrat> right
[Thursday 02 September 2010] [08:54:34] <sustrik> how come that Ctrl+C works then
[Thursday 02 September 2010] [08:54:35] <sustrik> ?
[Thursday 02 September 2010] [08:54:42] <sustrik> for C programs?
[Thursday 02 September 2010] [08:55:01] <sustrik> if main thread was stuck in a blocking call
[Thursday 02 September 2010] [08:55:02] <mato> ah, the thing is multiple threads are involved
[Thursday 02 September 2010] [08:55:08] <mato> this is what makes it tricky
[Thursday 02 September 2010] [08:55:25] <sustrik> but how does it work in C world?
[Thursday 02 September 2010] [08:55:27] <mato> ok, in the simple case
[Thursday 02 September 2010] [08:55:30] <mato> (1 thread)
[Thursday 02 September 2010] [08:55:40] <mato> the OS will invoke the default signal handler
[Thursday 02 September 2010] [08:55:46] <mato> which happens to be “exit the program”
[Thursday 02 September 2010] [08:56:10] <mato> the problem is, if you’re handling signals
[Thursday 02 September 2010] [08:56:12] <sustrik> ah, so no SIGINT
[Thursday 02 September 2010] [08:56:16] <sustrik> just hard exit
[Thursday 02 September 2010] [08:56:18] <mato> yes, you get SIGINT
[Thursday 02 September 2010] [08:56:23] <mato> it’s hard to explain...
[Thursday 02 September 2010] [08:56:32] <sustrik> try
[Thursday 02 September 2010] [08:56:46] * mato needs a whiteboard :)
[Thursday 02 September 2010] [08:56:52] <lestrrat> :)
[Thursday 02 September 2010] [08:56:54] <sustrik> ascii graphics!
[Thursday 02 September 2010] [08:57:12] <mikko> sustrik: it’s possible that i might be in Amsterdam whole of next week
[Thursday 02 September 2010] [08:57:22] <mikko> they are still trying to figure out schedules
[Thursday 02 September 2010] [08:57:28] <sustrik> aha
[Thursday 02 September 2010] [08:57:36] <sustrik> ok, we’ll see
[Thursday 02 September 2010] [08:57:37] <mato> ok, so, case 0: 1 thread, no signal handler: result: on ^C default “exit the program” handler is called by the OS
[Thursday 02 September 2010] [08:58:38] <mato> case 1: 1 thread, and a signal handler in that thread: result: on ^C, threads SIGINT handler is called and when that returns the blocking call it interrupted returns EINTR
[Thursday 02 September 2010] [08:58:50] <mato> it is now up to the program to resolve that EINTR
[Thursday 02 September 2010] [08:58:59] <mato> if it did not exit in the signal handler already, of course
[Thursday 02 September 2010] [08:59:15] <sustrik> ack
[Thursday 02 September 2010] [08:59:27] <mato> now, here’s what i imagine most language runtimes do in a signal handler
[Thursday 02 September 2010] [08:59:45] <mato> and in fact most apps with old-style (no separate thread with sigwait()) signal handlers
[Thursday 02 September 2010] [08:59:51] <mato> they set some flag
[Thursday 02 September 2010] [08:59:54] <mato> that’s all :)
[Thursday 02 September 2010] [09:00:14] <mato> hence, when the call returns EINTR, the app/Perl/whatever checks it’s “was i interrupted” flag
[Thursday 02 September 2010] [09:00:27] <mato> and if it was, deals with that in it’s normal control flow
[Thursday 02 September 2010] [09:00:36] <lestrrat> right
[Thursday 02 September 2010] [09:00:44] <sustrik> ok
[Thursday 02 September 2010] [09:00:46] <mato> now, enter 0mq
[Thursday 02 September 2010] [09:00:54] <mato> the i/o threads ignore all signals
[Thursday 02 September 2010] [09:00:56] * sustrik is thrilled
[Thursday 02 September 2010] [09:01:05] <sustrik> right
[Thursday 02 September 2010] [09:01:06] <mato> they may or may not get an EINTR if the process gets a signal
[Thursday 02 September 2010] [09:01:10] <mato> that is entirely OS-dependent
[Thursday 02 September 2010] [09:01:44] <mato> assuming for a moment that they did, the naive implementation would just return that EINTR, which would bubble up to the app thread logic and something useful might happen
[Thursday 02 September 2010] [09:01:53] <mato> however, that is a bad assumption
[Thursday 02 September 2010] [09:02:19] <mato> so, what to do?
[Thursday 02 September 2010] [09:02:20] <sustrik> wait a sec
[Thursday 02 September 2010] [09:02:28] * mato waiting
[Thursday 02 September 2010] [09:02:31] <sustrik> are you saying that I/O threadsmay get the signal;?
[Thursday 02 September 2010] [09:02:45] <mato> I/O threads will never “get” the signal
[Thursday 02 September 2010] [09:02:49] <sustrik> ok
[Thursday 02 September 2010] [09:03:03] <mato> but depending on your OS, they may get an EINTR back from any random syscall
[Thursday 02 September 2010] [09:03:09] <sustrik> ack
[Thursday 02 September 2010] [09:03:14] <mato> well, not any random, but anything that does something complex
[Thursday 02 September 2010] [09:03:21] <mato> i.e. not getpid() but send() is a candidate
[Thursday 02 September 2010] [09:03:24] <sustrik> ok, but that’s beside the point
[Thursday 02 September 2010] [09:03:31] <mato> it’s not
[Thursday 02 September 2010] [09:03:31] <sustrik> beause we are interested in app threads
[Thursday 02 September 2010] [09:03:35] <sustrik> not i/o threads
[Thursday 02 September 2010] [09:03:53] <sustrik> whether i/o thread gets sigint or not, we don’t care
[Thursday 02 September 2010] [09:04:19] <mato> yes we do, because if you were to be 100% compatible you’d have to “emulate” the behaviour i described in case 1
[Thursday 02 September 2010] [09:04:32] <mato> s/compatible/nice to broken apps/interpreters/
[Thursday 02 September 2010] [09:04:46] <sustrik> wait
[Thursday 02 September 2010] [09:04:54] <sustrik> i/o thread just loops
[Thursday 02 September 2010] [09:04:55] <lestrrat> isn’t thre recv() being called on the app thread?
[Thursday 02 September 2010] [09:05:08] <sustrik> so if it gets EINTR it can just ignore it
[Thursday 02 September 2010] [09:05:25] <mato> hmm, yes
[Thursday 02 September 2010] [09:05:30] <sustrik> lestrrat: that’s my point
[Thursday 02 September 2010] [09:05:49] <mato> ok, right, so you want the app thread side of the API to pass EINTR
[Thursday 02 September 2010] [09:05:54] <sustrik> ack
[Thursday 02 September 2010] [09:05:56] <lestrrat> right
[Thursday 02 September 2010] [09:06:18] <mato> hmm
[Thursday 02 September 2010] [09:06:33] <mato> ok, one way to do it w/o breaking compatibility
[Thursday 02 September 2010] [09:06:36] <mato> with existing code
[Thursday 02 September 2010] [09:06:49] <mato> is a context option
[Thursday 02 September 2010] [09:06:57] <mato> “ZMQ_INTERRUPTIBLE”
[Thursday 02 September 2010] [09:07:09] <sustrik> ok, so it would work?
[Thursday 02 September 2010] [09:07:11] <mato> meaning “API calls will return EINTR if interrupted by a signal”
[Thursday 02 September 2010] [09:07:19] <mato> i think so
[Thursday 02 September 2010] [09:07:24] <sustrik> woohoo
[Thursday 02 September 2010] [09:07:30] <mato> note that there is no change on the i/o thread side
[Thursday 02 September 2010] [09:07:35] <lestrrat> coolness
[Thursday 02 September 2010] [09:07:52] <sustrik> or a compile time option
[Thursday 02 September 2010] [09:07:59] <mato> no, compile time is bad
[Thursday 02 September 2010] [09:08:04] <mato> libzmq.so may be shared
[Thursday 02 September 2010] [09:08:06] <lestrrat> I’d vote for ZMQ_INTERRUPTABLE
[Thursday 02 September 2010] [09:08:17] <sustrik> there are no context-wide options
[Thursday 02 September 2010] [09:08:26] <mato> well, they’ll have to be added then :)
[Thursday 02 September 2010] [09:08:40] <sustrik> but this is a hack
[Thursday 02 September 2010] [09:08:44] <mato> ?
[Thursday 02 September 2010] [09:08:57] <sustrik> the right solution is tu return EINTR
[Thursday 02 September 2010] [09:09:22] <sustrik> we need the option only to stay backward compatible with the original lousy solution
[Thursday 02 September 2010] [09:09:39] <mato> hmm hmm
[Thursday 02 September 2010] [09:09:49] <sustrik> i don’t like changing API because of a hack
[Thursday 02 September 2010] [09:10:01] <mato> sustrik: just a minute, i’m still thinking
[Thursday 02 September 2010] [09:10:06] <lestrrat> I was wondering from the beginning, but does it have to “return” EINTR? is it not enough to keep errno = EINTR ?
[Thursday 02 September 2010] [09:10:24] <sustrik> errno = EINTR
[Thursday 02 September 2010] [09:10:26] <mato> lestrrat: yes, we mean return (-1) with errno = EINTR
[Thursday 02 September 2010] [09:10:33] <lestrrat> ah
[Thursday 02 September 2010] [09:10:39] <sustrik> but still it breaks the API specification
[Thursday 02 September 2010] [09:10:54] <sustrik> EINTR is not described as valid error from recv()
[Thursday 02 September 2010] [09:11:04] <mato> recv() or zmq_recv()?
[Thursday 02 September 2010] [09:11:19] <sustrik> zmq_recv(), sorry
[Thursday 02 September 2010] [09:11:30] <mato> so we’ll have to change that, i think
[Thursday 02 September 2010] [09:11:38] <sustrik> people will hate us
[Thursday 02 September 2010] [09:11:52] <mato> then add ZMQ_INTERRUPTIBLE
[Thursday 02 September 2010] [09:11:57] <sustrik> :)
[Thursday 02 September 2010] [09:12:06] <lestrrat> hey, I’m willing to accept a zmq_recv2() ;P
[Thursday 02 September 2010] [09:12:07] <mato> the thing is, the model of never getting EINTR is actually right if you do your handling properly
[Thursday 02 September 2010] [09:12:28] <mato> kind of
[Thursday 02 September 2010] [09:12:33] <sustrik> shrug
[Thursday 02 September 2010] [09:12:48] <lestrrat> does that mean the “correct” way is for me to install a sighandler in my binding?
[Thursday 02 September 2010] [09:13:08] <mato> lestrrat: that probably won’t help because of the way the interpreter is architected
[Thursday 02 September 2010] [09:13:22] <mato> lestrrat: but yes, that would be the correct way
[Thursday 02 September 2010] [09:13:29] <mato> in an ideal world :)
[Thursday 02 September 2010] [09:13:33] <sustrik> in broader sense i would say: let’s keep with POSIX API
[Thursday 02 September 2010] [09:13:45] <sustrik> trying to outsmart it just causes problems
[Thursday 02 September 2010] [09:13:45] <lestrrat> oh yeah, recv is a currently a loop, ain’t it...
[Thursday 02 September 2010] [09:13:58] <mato> sustrik: well, what this requires then is...
[Thursday 02 September 2010] [09:14:10] <mato> sustrik: a nice thorough explanation by email
[Thursday 02 September 2010] [09:14:16] <mato> sustrik: combined with “sorry, we messed up”
[Thursday 02 September 2010] [09:14:23] <sustrik> we did
[Thursday 02 September 2010] [09:14:27] <mato> sustrik: and it’ll just go into 2.1
[Thursday 02 September 2010] [09:14:37] <mato> eventually, not immediately
[Thursday 02 September 2010] [09:14:58] <mato> after actual verification with at least say Perl, Python, Ruby that it does solve the problem
[Thursday 02 September 2010] [09:15:18] <sustrik> actually, it was brian granger who asked for backward compatibility guarantees :)
[Thursday 02 September 2010] [09:15:27] <lestrrat> would it be acceptable to add a different, aptly named function?
[Thursday 02 September 2010] [09:15:41] <lestrrat> zmq_recv_intr() or whatever.
[Thursday 02 September 2010] [09:15:41] <mato> lestrrat: no, because it involves ALL API calls that can block
[Thursday 02 September 2010] [09:15:53] <lestrrat> hmm
[Thursday 02 September 2010] [09:16:07] <mato> it is actually a mistake on our part, but i didn’t realise until today why it was a mistake
[Thursday 02 September 2010] [09:16:32] <mato> well, we could be anal and say “make your signal handling work like this”, but that’s impossible in the real world
[Thursday 02 September 2010] [09:17:02] <sustrik> ok, let me write an email describing the problem
[Thursday 02 September 2010] [09:17:11] <mato> do you understand it well enough?
[Thursday 02 September 2010] [09:17:21] <mato> this email must not be compressed :-)
[Thursday 02 September 2010] [09:17:24] <sustrik> and asking whether breaking backward compatibility is acceptable in this case
[Thursday 02 September 2010] [09:17:25] <sustrik> no
[Thursday 02 September 2010] [09:17:29] <sustrik> you can write it
[Thursday 02 September 2010] [09:17:44] <sustrik> but it involves saying “i am an idiot, sorry”
[Thursday 02 September 2010] [09:17:49] <mato> i’d like to verify that it actually helps
[Thursday 02 September 2010] [09:18:00] <mato> yes, true, you were the one that defined the behaviour
[Thursday 02 September 2010] [09:18:14] <mato> ok, look, i’ll write the text describing the problem
[Thursday 02 September 2010] [09:18:23] <mato> you can send it out, adding “I’m an idiot”
[Thursday 02 September 2010] [09:18:27] <mato> ok? :)
[Thursday 02 September 2010] [09:18:27] <sustrik> :)
[Thursday 02 September 2010] [09:18:41] <sustrik> ok, let’s first check whether it helps
[Thursday 02 September 2010] [09:18:44] <mato> but before you publicy denounce yourself, it would be nice to check first
[Thursday 02 September 2010] [09:18:46] <mato> precisely
[Thursday 02 September 2010] [09:18:58] <mato> sustrik: i would suggest being very pendantic about this
[Thursday 02 September 2010] [09:19:00] <sustrik> if it turns out that it does not i don’t have to call myself an idiotr
[Thursday 02 September 2010] [09:19:12] <mato> i.e. make three test cases (Perl, Python, Ruby)
[Thursday 02 September 2010] [09:19:17] <sustrik> ok, i can create a topic branch
[Thursday 02 September 2010] [09:19:19] <mato> ensure they hang currently
[Thursday 02 September 2010] [09:19:21] <sustrik> fix it there
[Thursday 02 September 2010] [09:19:24] <mato> i.e. ^C doesn’t work
[Thursday 02 September 2010] [09:19:31] <mato> then make your change
[Thursday 02 September 2010] [09:19:36] <mato> and ensure that all works as expected
[Thursday 02 September 2010] [09:19:55] <mato> sustrik: yes, topic branch, involves doc changes and so on
[Thursday 02 September 2010] [09:19:57] <mato> good idea
[Thursday 02 September 2010] [09:20:11] <sustrik> ok, let me do it
[Thursday 02 September 2010] [09:20:14] <lestrrat> let me know when I can test it :)
[Thursday 02 September 2010] [09:20:19] <mato> sustrik: i will find the magic command for you to email the topic branch patch set around
[Thursday 02 September 2010] [09:20:22] <sustrik> lestrrat: i’ll ping you
[Thursday 02 September 2010] [09:20:30] <sustrik> i’ll ping brian as well
[Thursday 02 September 2010] [09:20:32] <mato> sustrik: so that you can give it to e.g. lestrrat
[Thursday 02 September 2010] [09:20:33] <mato> ja
[Thursday 02 September 2010] [09:20:46] <sustrik> he’s willing to test it, we’ve discussed it in the morning
[Thursday 02 September 2010] [09:20:54] <mato> just email the patches around privately if you don’t want to call yourself a potential idiot just yet :)
[Thursday 02 September 2010] [09:20:56] <sustrik> not sure about ruby
[Thursday 02 September 2010] [09:20:58] <mato> (in public) :)
[Thursday 02 September 2010] [09:21:03] <sustrik> cremes: are you here?
[Thursday 02 September 2010] [09:21:53] <lestrrat> +1 for branch (just easier to pull ;)
[Thursday 02 September 2010] [09:22:13] <sustrik> lestrrat: what’s you email
[Thursday 02 September 2010] [09:22:18] <sustrik> how should i ping you?
[Thursday 02 September 2010] [09:22:24] <sustrik> thx
[Thursday 02 September 2010] [09:22:30] <lestrrat> lestrrat @ github, lestrrat @ twitter
[Thursday 02 September 2010] [09:22:34] <sustrik> ok
[Thursday 02 September 2010] [09:22:34] <mato> lestrrat: shhh... i’m slowly teaching sustrik git
[Thursday 02 September 2010] [09:22:42] <lestrrat> lol
[Thursday 02 September 2010] [09:22:52] <mato> start with local branches
[Thursday 02 September 2010] [09:22:53] <mato> :)
[Thursday 02 September 2010] [09:24:07] <sustrik> ok
[Thursday 02 September 2010] [09:27:28] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [09:29:01] <mato> ok, i have to concentrate on something else for a bit
[Thursday 02 September 2010] [09:29:12] <mato> ping me if you need me, bbl
[Thursday 02 September 2010] [09:31:39] <sustrik> cya
[Thursday 02 September 2010] [09:53:57] <cremes> sustrik: just got here; what do you need?
[Thursday 02 September 2010] [09:54:25] <sustrik> are you seeing problem with Ctrl+C in Ruby?
[Thursday 02 September 2010] [09:54:46] <sustrik> i mean, application not responding to SIGINT?
[Thursday 02 September 2010] [09:55:10] <sustrik> when stuck in 0mq blocking call?
[Thursday 02 September 2010] [09:55:13] <cremes> yes
[Thursday 02 September 2010] [09:55:24] <sustrik> aha, good
[Thursday 02 September 2010] [09:55:31] <sustrik> we think we’ve found a solution
[Thursday 02 September 2010] [09:55:32] <cremes> i don’t think the ruby signal handler runs when external C code is executing
[Thursday 02 September 2010] [09:55:39] <cremes> really? that’s good
[Thursday 02 September 2010] [09:55:54] <sustrik> would you be willing to test it once i have a fix?
[Thursday 02 September 2010] [09:56:33] <sustrik> the idea is that the blocking calls would return EINTR in case of Ctrl+C
[Thursday 02 September 2010] [09:56:57] <sustrik> then the binding can take care or what happens next
[Thursday 02 September 2010] [09:57:11] <cremes> sure
[Thursday 02 September 2010] [09:57:20] <sustrik> great, i’ll ping you once i have it
[Thursday 02 September 2010] [09:57:25] <cremes> ok
[Thursday 02 September 2010] [10:05:49] <mrm2m> ignore those random and threading they are not used.
[Thursday 02 September 2010] [10:06:41] <mrm2m> the “thread started” shows up, but nothing is sent to the corresponding server.
[Thursday 02 September 2010] [10:09:48] <sustrik> mrm2m: the application exits after sending the message?
[Thursday 02 September 2010] [10:10:00] <mrm2m> right
[Thursday 02 September 2010] [10:10:30] <sustrik> in 2.0.8 all the unsent data are discarded when socket is closed
[Thursday 02 September 2010] [10:10:54] <mrm2m> Uh - ok
[Thursday 02 September 2010] [10:11:19] <sustrik> the semantics is changed in 2.1
[Thursday 02 September 2010] [10:11:39] <sustrik> it’s: block context termination while all data are sent
[Thursday 02 September 2010] [10:12:23] <guido_g> so you can’t exit if something is broken?
[Thursday 02 September 2010] [10:12:44] <sustrik> it’s annoying, i know
[Thursday 02 September 2010] [10:13:02] <sustrik> what i want to do is to add SO_LINGER socket option
[Thursday 02 September 2010] [10:13:12] <lestrrat> would be nice if it was configurable
[Thursday 02 September 2010] [10:13:18] <guido_g> ack
[Thursday 02 September 2010] [10:13:18] <sustrik> with same semantics as with POSIX sockets
[Thursday 02 September 2010] [10:13:42] <sustrik> that should do imo
[Thursday 02 September 2010] [10:16:23] Quit bgranger has left this server (Quit: bgranger).
[Thursday 02 September 2010] [11:07:11] Quit poswald has left this server (Quit: poswald).
[Thursday 02 September 2010] [11:10:07] Quit mrm2m has left this server (Quit: Leaving.).
[Thursday 02 September 2010] [11:11:30] Quit poswald has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [11:25:43] <gavinstark> I am following the PUB/SUB example in the ‘guide’ but I’m wondering how I might setup a way for there to be multiple publishers that each subscriber can receive from. Or do I need independent bind/connects for that?
[Thursday 02 September 2010] [11:27:54] <mikko> gavinstark: the latter
[Thursday 02 September 2010] [11:34:29] <cremes> gavinstark: you also need a forwarder device to aggregate all of the publisher’s messages
[Thursday 02 September 2010] [11:35:01] <cremes> (not strictly true, but i think it’s the cleanest way to set things up)
[Thursday 02 September 2010] [11:42:47] <gavinstark> Do I have to pre-list all the publishers in the forwarder config? It seems like I’d have to have multiple <in> with unique values for each publisher that might appear? Or am I missing something?
[Thursday 02 September 2010] [11:55:00] <sustrik> gavinstark: bind your forwarder devices
[Thursday 02 September 2010] [11:55:12] <sustrik> connect the publishers and subscribers
[Thursday 02 September 2010] [11:57:42] <gavinstark> sustrik, what if I do not know the qty of publishers before hand? Won’t each publisher have to bind uniquely? (
tcp://....:5555,
tcp://....:5556, etc.?)
[Thursday 02 September 2010] [11:59:07] <sustrik> pubishers should _connect_ to the forwared
[Thursday 02 September 2010] [11:59:11] <sustrik> forwarder*
[Thursday 02 September 2010] [12:06:44] <gavinstark> surstrik, ah, ok. I just tried that, having the publisher zmq_connect instead of bind, still not quite working. Here is what I did:
[Thursday 02 September 2010] [12:08:10] <gavinstark> ah, I think I see, I was supposed to “bind” on the “in” entry?
[Thursday 02 September 2010] [12:12:45] <sustrik> gavinstark: forwarder should bind both in and out
[Thursday 02 September 2010] [12:12:47] <zedas> lestrrat: re: mongrel2 handlers blocking, i fixed it by making the zeromq IO threads be 1. I’ve been running it like that for weeks without any problems, so you should be fine.
[Thursday 02 September 2010] [12:13:12] <gavinstark> sustrik: Thanks, working perfectly now.
[Thursday 02 September 2010] [12:13:15] <sustrik> zedas: this is a different issue
[Thursday 02 September 2010] [12:13:50] <sustrik> annoying interactions between language runime, OS signals and 0MQ async architecture
[Thursday 02 September 2010] [12:15:02] <zedas> sustrik: ah. you got a link for me about it?
[Thursday 02 September 2010] [12:15:04] <mato> that’s a very polite way of putting it :-)
[Thursday 02 September 2010] [12:15:54] <sustrik> mato: are you able to summarise it for zed?
[Thursday 02 September 2010] [12:16:00] <mato> yeah
[Thursday 02 September 2010] [12:16:04] * sustrik feels incompetent here
[Thursday 02 September 2010] [12:16:08] <mato> signal handling is fucked :-)
[Thursday 02 September 2010] [12:16:11] <mato> end of summary
[Thursday 02 September 2010] [12:16:32] <sustrik> up to the point
[Thursday 02 September 2010] [12:17:04] <sustrik> basically, it has to do with handling Ctrl+C in interpreted languages
[Thursday 02 September 2010] [12:17:07] <mato> to elaborate on my summary, the issue is with the 0mq API not returning EINTR when API calls are interrupted by a signal in the application thread
[Thursday 02 September 2010] [12:17:09] <sustrik> i’m working on a fix now
[Thursday 02 September 2010] [12:17:23] <mato> zedas: problem is most language runtimes do delayed signal handling
[Thursday 02 September 2010] [12:17:33] <mato> zedas: i.e. handler() just sets some flag
[Thursday 02 September 2010] [12:17:39] <mato> zedas: it doesn’t actually do anything
[Thursday 02 September 2010] [12:17:46] <mato> zedas: flag gets picked up when the runtime wakes up
[Thursday 02 September 2010] [12:18:01] <mato> zedas: but since 0mq calls never return EINTR, runtime never wakes up
[Thursday 02 September 2010] [12:22:32] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Thursday 02 September 2010] [12:25:08] Quit keffo has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [12:26:22] <zedas> mato: ah yes, that’d explain it.
[Thursday 02 September 2010] [12:35:20] <ModusPwnens> hi, i’m back again. Hopefully with a question that’s not as dumb this time. Anyways, does anyone know what would cause the first two bytes of a message to get corrupted?
[Thursday 02 September 2010] [12:39:20] <bgranger> zedas: can you say more about what you have been doing to solve this. I was going to look into the EINTR stuff today.
[Thursday 02 September 2010] [12:40:19] <mato> bgranger: I think we have a solution for the EINTR stuff
[Thursday 02 September 2010] [12:40:27] <sustrik> ModusPwnens: do you have a test program?
[Thursday 02 September 2010] [12:40:46] <bgranger> mato: ?
[Thursday 02 September 2010] [12:40:48] <mato> bgranger: oh, hang on, maybe you mean zed’s issue
[Thursday 02 September 2010] [12:40:56] <bgranger> Is that not the same thing?
[Thursday 02 September 2010] [12:41:07] <mato> as opposed to the general issue which manifests itself as “^C doesn’t work in $RANDOM_LANGUAGE”
[Thursday 02 September 2010] [12:41:09] <bgranger> I missed some of the discussion so I am trying to piece it together
[Thursday 02 September 2010] [12:41:14] <mato> ah
[Thursday 02 September 2010] [12:41:24] <bgranger> Are there 2 different issues?
[Thursday 02 September 2010] [12:41:27] <mato> yeah
[Thursday 02 September 2010] [12:41:42] <mato> zedas has an issue with multiple i/o threads
[Thursday 02 September 2010] [12:41:53] <mato> which may or may not have something (different) to do with EINTR
[Thursday 02 September 2010] [12:42:00] <mato> not clear, haven’t had time to look at that
[Thursday 02 September 2010] [12:42:00] <bgranger> And signals?
[Thursday 02 September 2010] [12:42:03] <mato> signals
[Thursday 02 September 2010] [12:42:11] <mato> The problem is as I described just above.
[Thursday 02 September 2010] [12:42:18] <bgranger> What is the idea about EINTR?
[Thursday 02 September 2010] [12:42:33] <mato> When Martin Sustrik made the original API he decided not to return EINTR from blocking calls
[Thursday 02 September 2010] [12:42:39] <bgranger> I did talk to sustrik late last night (for me) and he pointed me to some code ...
[Thursday 02 September 2010] [12:42:46] <bgranger> Right he showed me that code
[Thursday 02 September 2010] [12:43:01] <mato> Right, except that basically breaks standard signal handling
[Thursday 02 September 2010] [12:43:13] <bgranger> Today I am going to put in a print statement in that logic and try to see if it prints with a SIGINT in the Python bindings.
[Thursday 02 September 2010] [12:43:42] <bgranger> It is still not clear if this will work with the Python bindings, but we will see.
[Thursday 02 September 2010] [12:43:43] <mato> To recap, imagine the simplest case with a C program blocking on some syscall, while at the same time handling say SIGINT.
[Thursday 02 September 2010] [12:43:44] <bgranger> I think it may
[Thursday 02 September 2010] [12:43:49] <ModusPwnens> sustrik: Yeah, it’s happening in my code. I’m not entirely sure why. It seems like it happens when it is sent, as the sending side can properly decode the message. However, the receving side cannot because the first two bytes get corrupted for some reason. I can paste my code to a pastebin if you like.
[Thursday 02 September 2010] [12:44:22] <mato> bgranger: Now, if “handling” SIGINT in this programs case means it just prints “Interrupted!” and exits, then fine.
[Thursday 02 September 2010] [12:44:33] <mato> bgranger: That will work even with the current situation in 0MQ.
[Thursday 02 September 2010] [12:44:53] <bgranger> Not in the python bindings...
[Thursday 02 September 2010] [12:45:10] <mato> bgranger: But, if it instead means that the program just sets some random flag, and then expects to process that flag “later”, it won’t work.
[Thursday 02 September 2010] [12:45:19] <bgranger> Right
[Thursday 02 September 2010] [12:45:27] <mato> bgranger: Which is precisely the Python/Perl (at least) case
[Thursday 02 September 2010] [12:45:32] <bgranger> Which is what Python does...yep
[Thursday 02 September 2010] [12:45:55] <sustrik> ModusPwnens: try it
[Thursday 02 September 2010] [12:45:56] <mato> bgranger: So, the only real solution is that 0MQ API calls return EINTR if they get EINTR back from a blocking system call.
[Thursday 02 September 2010] [12:46:12] <bgranger> Will that happen regardless of what signal handlers have been installed?
[Thursday 02 September 2010] [12:46:19] <mato> Yup
[Thursday 02 September 2010] [12:46:20] <ModusPwnens> sustrik: do you want the entire code or just the functions in question?
[Thursday 02 September 2010] [12:46:36] <mato> bgranger: It’s what the OS does.
[Thursday 02 September 2010] [12:46:40] <bgranger> That would definittely solve our problems then! e would be very happy about that.
[Thursday 02 September 2010] [12:46:49] <sustrik> well, i would like a simple example
[Thursday 02 September 2010] [12:46:57] <sustrik> showing the problem
[Thursday 02 September 2010] [12:47:05] <mato> bgranger: If you do e.g. poll () in C, handle a signal in the same thread, then that poll () will return EINTR
[Thursday 02 September 2010] [12:47:19] <mato> bgranger: by “handle a signal” I mean the “set a flag case”
[Thursday 02 September 2010] [12:47:23] <sustrik> bgranger: i’ll fix it and let you know
[Thursday 02 September 2010] [12:47:24] <bgranger> mato: right OK
[Thursday 02 September 2010] [12:47:39] <sustrik> you can test it with python, others will test with perl and ruby
[Thursday 02 September 2010] [12:47:43] <mato> bgranger: By the way, while you’re here, have you had any more Python/GIL issues?
[Thursday 02 September 2010] [12:47:49] <bgranger> sustrik: can you fix it in the 2.0.8 branch. We are not using trunk yet
[Thursday 02 September 2010] [12:48:09] <sustrik> backwards compatibility :|
[Thursday 02 September 2010] [12:48:11] <bgranger> mato: No we have solved those. It was super sutble to get non-copy send/recv working with the GIL though.
[Thursday 02 September 2010] [12:48:20] <bgranger> mato: But it is working well.
[Thursday 02 September 2010] [12:48:24] <mato> bgranger: I have a really interesting case which looks like one of those...
[Thursday 02 September 2010] [12:48:44] <mato> bgranger: Will send to the mailing list, tomorrow.
[Thursday 02 September 2010] [12:49:06] <ModusPwnens> Hmm.well i don’t really have an example..it’s just happening in the code I have written, so i could give you that if you wanted to see it. It started happening after i used google protobufs to encode into a byte array instead of a string.
[Thursday 02 September 2010] [12:49:11] <bgranger> mato: Ok, I will watch for it. The challenge was getting the ref counts of zmq messages synch’d with those of Python.
[Thursday 02 September 2010] [12:49:30] <mato> bgranger: But the short story is, I see gc trying to close() a socket, while at the same time zmq_free_fn() in a different thread is trying to acquire the GIL in order to decrease the message refcount
[Thursday 02 September 2010] [12:49:39] <mato> bgranger: and the result is deadlock
[Thursday 02 September 2010] [12:50:01] <bgranger> Is this in trunk where sockets can move threads?
[Thursday 02 September 2010] [12:50:08] <mato> bgranger: nope, 2.0.8
[Thursday 02 September 2010] [12:50:22] <bgranger> What language?
[Thursday 02 September 2010] [12:50:27] <mato> Python...
[Thursday 02 September 2010] [12:50:35] <bgranger> What version of pyzmq?
[Thursday 02 September 2010] [12:50:55] <mato> latest-ish, let me check
[Thursday 02 September 2010] [12:51:07] <bgranger> Since Saturday?
[Thursday 02 September 2010] [12:51:10] <mato> ah, no
[Thursday 02 September 2010] [12:51:16] <mato> 2.0.7 pyzmq actually
[Thursday 02 September 2010] [12:51:22] <mato> with 2.0.8 zmq
[Thursday 02 September 2010] [12:51:23] <bgranger> I did a bunch of work then. I release a 2.0.7 stable release and master is now 2.0.8 cmpatible.
[Thursday 02 September 2010] [12:51:58] <bgranger> But, I believe that what you are saying it possible. The 1 problem we have is that the zmq_free_fn does have to acquire the GIL. If that can’t happen, you have trouble.
[Thursday 02 September 2010] [12:52:01] <jonrafkind> has anyone used zmq in a real-time game? mostly I just need low latency
[Thursday 02 September 2010] [12:52:13] <mato> bgranger: that’s precisely what I’m seeing
[Thursday 02 September 2010] [12:52:13] <bgranger> There is nothing we can do to get around this.
[Thursday 02 September 2010] [12:52:32] <bgranger> Can you just hold onto the socket ref to prevent gc?
[Thursday 02 September 2010] [12:52:44] <mato> Yes, but I have transient sockets in this application
[Thursday 02 September 2010] [12:52:49] <mato> So tons of fds get leaked
[Thursday 02 September 2010] [12:52:57] <bgranger> Hmmm, that might be tough
[Thursday 02 September 2010] [12:53:06] <bgranger> What do you mean by that?
[Thursday 02 September 2010] [12:53:22] <mato> Well, it’s precisely my workaround (holding onto the socket refs)
[Thursday 02 September 2010] [12:53:29] <bgranger> Ahh, OK.
[Thursday 02 September 2010] [12:53:45] <mato> But that means the underlying fds hang around, so eventually you’ll run out.
[Thursday 02 September 2010] [12:53:46] <bgranger> Is the socket that is being gc’s the one that is sending the msg though?
[Thursday 02 September 2010] [12:53:53] <bgranger> right, you don’t want that
[Thursday 02 September 2010] [12:54:10] <mato> That’s hard to tell at the moment, but at least you’ve confirmed that this can happen.
[Thursday 02 September 2010] [12:54:18] <mato> So the backtraces I see make sense.
[Thursday 02 September 2010] [12:54:22] <mato> I’ll write it up tomorrow.
[Thursday 02 September 2010] [12:54:39] <sustrik> ModusPwnens: if you want me to look at it, strip it down to the simplest possible example that reproduces the bahviour
[Thursday 02 September 2010] [12:54:53] <bgranger> I can do the following. When a socket send a msg, it can add itself to a list of sockets that the Message hold on to. That way the message can prevent the gc, but when the msg goes away, the socket will as well.
[Thursday 02 September 2010] [12:55:09] <sustrik> ModusPwnens: aren’t you overwriting the buffer you’ve sent to 0MQ?
[Thursday 02 September 2010] [12:55:30] <bgranger> but that might keep socket around longer than you want if you have a message that love a long time.
[Thursday 02 September 2010] [12:55:46] <mato> bgranger: That might be a solution, yes.
[Thursday 02 September 2010] [12:55:54] <mato> bgranger: Anyhow, gc is not supposed to be instant, no?
[Thursday 02 September 2010] [12:56:02] <bgranger> Depends
[Thursday 02 September 2010] [12:56:13] <bgranger> if there are cycles or not.
[Thursday 02 September 2010] [12:56:18] <mato> bgranger: So if the socket hangs around for a bit... does it matter too much? As long as it goes away eventually.
[Thursday 02 September 2010] [12:56:30] <sustrik> jonrafkind: i think there are couple of game devs here, rbraley for example
[Thursday 02 September 2010] [12:56:35] <bgranger> Depends on what “a bit” means
[Thursday 02 September 2010] [12:56:48] <mato> True.
[Thursday 02 September 2010] [12:56:53] <ModusPwnens> Sustrik: Well it’s in a loop and I close the message at the end of the loop, initializing it again at the beginning
[Thursday 02 September 2010] [12:57:07] <mato> bgranger: Let me sleep on it, and write up, this is useful to have on the list
[Thursday 02 September 2010] [12:57:12] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [12:57:20] <mato> bgranger: since other people dealing with GC-based languages may run into similar problems.
[Thursday 02 September 2010] [12:57:22] <bgranger> mato: Great
[Thursday 02 September 2010] [12:57:26] <bgranger> Aboslutely
[Thursday 02 September 2010] [12:57:33] <ModusPwnens> and the data that I am initializing it with is a char * which i free at the end of each iteration too
[Thursday 02 September 2010] [12:57:55] <sustrik> show me the sending code
[Thursday 02 September 2010] [12:58:10] <mato> bgranger: Thanks for your help
[Thursday 02 September 2010] [12:58:25] * mato is off to dinner, see you all later
[Thursday 02 September 2010] [12:58:56] <bgranger> later
[Thursday 02 September 2010] [13:00:11] <ModusPwnens> there’s a lot of debugging stuff in there so i’m sorry that it is messy
[Thursday 02 September 2010] [13:00:11] <sustrik> ModusPwnens: when using zmq_msg_init_data you are passing ownership of the buffer to 0MQ
[Thursday 02 September 2010] [13:00:23] <sustrik> so you have to give it a free function
[Thursday 02 September 2010] [13:00:31] <sustrik> and don’t touch the buffer afterwards
[Thursday 02 September 2010] [13:01:12] <sustrik> if you don’t need zero-copy
[Thursday 02 September 2010] [13:01:22] <sustrik> just init the message using zmq_msg_init_size
[Thursday 02 September 2010] [13:01:29] <sustrik> and copt the data into i
[Thursday 02 September 2010] [13:01:30] <sustrik> it
[Thursday 02 September 2010] [13:01:59] <ModusPwnens> So just don’t use init_data at all?
[Thursday 02 September 2010] [13:02:50] <sustrik> do you need zero-copy?
[Thursday 02 September 2010] [13:03:05] <ModusPwnens> I’m not really sure what that is, so I don’t think so.
[Thursday 02 September 2010] [13:03:14] <sustrik> than don’t use it :)
[Thursday 02 September 2010] [13:03:30] <ModusPwnens> Ok! So just use size to initialize it and then memcpy into it?
[Thursday 02 September 2010] [13:03:36] <sustrik> exactly
[Thursday 02 September 2010] [13:03:42] <ModusPwnens> okie doke. Thanks! I will try that!
[Thursday 02 September 2010] [13:03:48] <ModusPwnens> Sorry for the constant questions :S
[Thursday 02 September 2010] [13:04:29] <sustrik> np
[Thursday 02 September 2010] [13:06:09] <jonrafkind> oh i just realied, imatrix is the same company that made SFL. I use that in my project :p
[Thursday 02 September 2010] [13:33:45] Quit gebi has left this server (Ping timeout: 272 seconds).
[Thursday 02 September 2010] [14:02:40] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [14:19:52] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [14:30:16] Join gebi has joined this channel (
~gebi@84.119.80.149).
[Thursday 02 September 2010] [15:22:39] Quit plq has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:43:34] Quit
snerd_ has left this server (Changing host).
[Thursday 02 September 2010] [15:47:33] Quit snerd has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:47:35] Quit impl has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:57:13] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [16:27:52] Quit xla has left this server (Quit: leaving).
[Thursday 02 September 2010] [16:29:01] Quit ModusPwnens has left this server.
[Thursday 02 September 2010] [16:50:50] Quit gavinstark has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [16:50:52] <ModusPwnens> Hi guys, I have encountered a problem with the official benchmarking utility.
[Thursday 02 September 2010] [16:51:31] <ModusPwnens> It appears to crash if you enter in a very large number of messages, and I was w ndering if this was supposed to happen and if so, why?
[Thursday 02 September 2010] [16:58:49] Nick
impl_ is now known as impl.
[Thursday 02 September 2010] [17:16:55] <cremes> ModusPwnens: which benchmarking utility and what number did you pass to it?
[Thursday 02 September 2010] [17:17:41] <ModusPwnens> The site has changed so I don’t know where the utility is offhand anymore
[Thursday 02 September 2010] [17:18:09] <cremes> what’s the name of it?
[Thursday 02 September 2010] [17:18:31] <ModusPwnens> actually, it’s in my zeromq folder
[Thursday 02 September 2010] [17:18:35] <ModusPwnens> in the bin
[Thursday 02 September 2010] [17:18:38] <ModusPwnens> remote_thr
[Thursday 02 September 2010] [17:18:52] <cremes> oh, the local_thr/remote_thr pair?
[Thursday 02 September 2010] [17:18:55] <ModusPwnens> ya
[Thursday 02 September 2010] [17:19:17] <cremes> so what arguments did you pass it? (i recommend you pastie the output from your shell along with any displayed error)
[Thursday 02 September 2010] [17:19:32] <ModusPwnens> i passed 50 and 25000000
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> C:UsersDavid DawsonDesktopzeromq-2.0.7zeromq-2.0.7bin>remote_thr.exe tcp:/
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> Assertion failed: end_chunk->next (c:usersdavid dawsondesktopzeromq-2.0.7ze
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> romq-2.0.7srcyqueue.hpp:108)
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> This application has requested the Runtime to terminate it in an unusual way.
[Thursday 02 September 2010] [17:20:01] <ModusPwnens> Please contact the application’s support team for more information.
[Thursday 02 September 2010] [17:20:32] <cremes> did it fail only with the 25 million number or with 50 too?
[Thursday 02 September 2010] [17:20:34] <ModusPwnens> bah, sorry, the command prompt is strange
[Thursday 02 September 2010] [17:20:42] <ModusPwnens> no, it’s just the 25 million
[Thursday 02 September 2010] [17:20:50] <ModusPwnens> if i lower it it works fine
[Thursday 02 September 2010] [17:20:58] <ModusPwnens> but I was just curious if it is supposed to fail that way
[Thursday 02 September 2010] [17:21:07] <cremes> no, you may have found a bug
[Thursday 02 September 2010] [17:21:16] <cremes> but before reporting it, install 2.0.8 and try again
[Thursday 02 September 2010] [17:21:18] <ModusPwnens> ok
[Thursday 02 September 2010] [17:21:22] <cremes> no sense in reporting a bug against an old release
[Thursday 02 September 2010] [17:21:32] <ModusPwnens> true enough
[Thursday 02 September 2010] [17:22:10] <ModusPwnens> ok i have to recompile the source, hold on
[Thursday 02 September 2010] [17:22:34] <cremes> just a note on irc etiquette...
[Thursday 02 September 2010] [17:22:44] <cremes> give as much information as possible...
[Thursday 02 September 2010] [17:23:11] <cremes> don’t paste more than 2 lines directly into the channel; use a pastie srevice like pastie.org or gist.github.com for longer stuff
[Thursday 02 September 2010] [17:23:25] <cremes> tell us the name of the programs involved and the version of the library
[Thursday 02 September 2010] [17:23:43] Quit dos000 has left this server (Read error: Connection reset by peer).
[Thursday 02 September 2010] [17:23:48] <ModusPwnens> Ok. I will note that for the future.
[Thursday 02 September 2010] [17:23:56] <ModusPwnens> thanks!
[Thursday 02 September 2010] [17:23:56] <cremes> if you see a crash, it is never supposed to happen so asking if it is seems a bit silly
[Thursday 02 September 2010] [17:24:06] <cremes> np
[Thursday 02 September 2010] [17:24:09] <ModusPwnens> Well, i guess I meant to say if it was already known
[Thursday 02 September 2010] [17:24:47] <cremes> then search the issues on github; all known bugs are reported and tracked there
[Thursday 02 September 2010] [17:25:08] <ModusPwnens> Hmm, ok. I didn’t know about that..
[Thursday 02 September 2010] [17:25:43] <cremes> and now you know! ;)
[Thursday 02 September 2010] [17:25:54] <ModusPwnens> that would be under the issues section?
[Thursday 02 September 2010] [17:26:12] <cremes> correct
[Thursday 02 September 2010] [17:26:34] <ModusPwnens> Okay. Sorry about that..
[Thursday 02 September 2010] [17:27:51] <cremes> we all had to learn it at some point, so don’t worry about it
[Thursday 02 September 2010] [17:28:13] Quit dos000 has left this server (Ping timeout: 258 seconds).
[Thursday 02 September 2010] [17:29:02] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 02 September 2010] [17:52:01] Quit gavinstark has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [18:03:02] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 02 September 2010] [18:31:17] <ModusPwnens> ok, so it still crashes
[Thursday 02 September 2010] [18:31:22] <ModusPwnens> i will paste the output in as ec
[Thursday 02 September 2010] [18:33:20] <ModusPwnens> I am using windows 7 on both computers
[Thursday 02 September 2010] [18:33:36] <ModusPwnens> The computer that generated the error is 64-bit as well
[Thursday 02 September 2010] [18:34:29] <ModusPwnens> the computer running local is only 32-bit
[Thursday 02 September 2010] [19:43:18] Quit poswald has left this server (Quit: poswald).
[Thursday 02 September 2010] [19:47:56] Quit poswald has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [20:07:47] <bgranger> sustrik: Just played with the EINTR logic.
[Thursday 02 September 2010] [20:31:13] Quit ModusPwnens has left this server.
[Thursday 02 September 2010] [20:35:38] Quit jonrafkind has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [21:25:39] Quit xrfang has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [21:46:33] <halfie> hi, does zeromq supports building of read-only queues with one writer and multiple readers?
[Thursday 02 September 2010] [21:47:38] Quit rossij has left this server (Ping timeout: 245 seconds).
[Thursday 02 September 2010] [22:10:13] Quit gebi has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [22:14:20] <halfie> ls
[Thursday 02 September 2010] [22:26:17] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [22:37:15] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:08:08] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:17:00] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:26:55] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:29:33] Quit rbraley has left this server (
*.net
*.split).
[Thursday 02 September 2010] [23:29:33] Quit GeekGod has left this server (
*.net
*.split).
[Thursday 02 September 2010] [23:38:49] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:48:43] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:56:19] Quit xrfang has left this server (Remote host closed the connection).
[Friday 03 September 2010] [00:31:22] Quit jsimmons has left this server (Quit: Leaving).
[Friday 03 September 2010] [00:39:25] Quit grahamalot has left this server (Quit: grahamalot).
[Friday 03 September 2010] [00:40:21] Join mikejs has joined this channel (
~me@mikej.st).
[Friday 03 September 2010] [01:51:52] Quit bgranger has left this server (Quit: bgranger).
[Friday 03 September 2010] [01:54:31] <guido_g> g’mornin’ (or whatever :)
[Friday 03 September 2010] [02:20:51] <sustrik> guido_g: thanks, i’ll have a look
[Friday 03 September 2010] [02:20:54] <sustrik> morning
[Friday 03 September 2010] [02:21:01] <sjampoo> I am kinda sad to see that with each update I find it more difficult to navigate around zeromq.??? :(
[Friday 03 September 2010] [02:22:05] <sustrik> sjampoo: i thought pieter reverted it to the original state
[Friday 03 September 2010] [02:22:05] <sustrik> let me see
[Friday 03 September 2010] [02:22:15] <sustrik> you mean the news should be in the right pane?
[Friday 03 September 2010] [02:22:28] <sjampoo> yes, at the least
[Friday 03 September 2010] [02:22:54] <sustrik> if you have any idea what to do further, just shoot
[Friday 03 September 2010] [02:24:21] <sjampoo> first thing would be not to change it every month ;) and I send an email about it months ago to the list. I also do not really like the .com / .org split
[Friday 03 September 2010] [02:24:49] <sjampoo> If the .org site is difficult to understand for new users, just make it more easy to understand
[Friday 03 September 2010] [02:25:04] <sustrik> where would you put the community stuff?
[Friday 03 September 2010] [02:25:20] <sjampoo> I would hide that on a separate but consistent page
[Friday 03 September 2010] [02:25:33] <sustrik> www.zeromq.org/dev say?
[Friday 03 September 2010] [02:25:42] <sjampoo> Yes, something like that.
[Friday 03 September 2010] [02:26:00] <sjampoo> I mean only 0.005% is interested what the actual last edited page is.
[Friday 03 September 2010] [02:26:22] <sjampoo> Most people interested in the community, will most likely just want to see GITHUB or the IssueList
[Friday 03 September 2010] [02:26:35] <sustrik> no mailing list?
[Friday 03 September 2010] [02:26:56] <sjampoo> and the mailinglist
[Friday 03 September 2010] [02:27:06] <sjampoo> But I don’t think that should be on the frontpage
[Friday 03 September 2010] [02:27:40] <sustrik> wouldn’t you mind clicking through a simple page each time to the dev page?
[Friday 03 September 2010] [02:28:20] <sjampoo> No, not if it is at a consistent location
[Friday 03 September 2010] [02:28:25] <sjampoo> I could bookmark it
[Friday 03 September 2010] [02:28:29] <sustrik> right
[Friday 03 September 2010] [02:28:42] <sustrik> in summary:
[Friday 03 September 2010] [02:28:50] <sustrik> 1. move zeromq.com to zeromq.org
[Friday 03 September 2010] [02:28:59] <sustrik> 2. move zeromq.org to zeromq.org/community
[Friday 03 September 2010] [02:29:10] <sustrik> 3. move the news to the right pane
[Friday 03 September 2010] [02:29:29] <sustrik> 4. get rid of recent site changes
[Friday 03 September 2010] [02:29:37] <sustrik> that’s it?
[Friday 03 September 2010] [02:30:15] <sjampoo> I am not completely sure what you mean with moving .com to .org.
[Friday 03 September 2010] [02:30:20] <sjampoo> I am not a big fan of the .com design
[Friday 03 September 2010] [02:30:36] <sustrik> that’s because you are part of community
[Friday 03 September 2010] [02:30:44] <sjampoo> No, no
[Friday 03 September 2010] [02:31:06] <sjampoo> I just don’t think that people landing on that site will get a good idea about what 0mq is
[Friday 03 September 2010] [02:31:20] <sustrik> ok, here’s the use case:
[Friday 03 September 2010] [02:31:21] <sjampoo> It needs a little more meat
[Friday 03 September 2010] [02:31:46] <sustrik> you have a problem and someone advises you 0mq may help to solve it
[Friday 03 September 2010] [02:31:55] <sustrik> you type zeromq into goodle
[Friday 03 September 2010] [02:31:58] <sustrik> google
[Friday 03 September 2010] [02:32:15] <sustrik> which, ideally, would bring you to what’s now zeromq.com
[Friday 03 September 2010] [02:32:27] <sustrik> you are under time stress
[Friday 03 September 2010] [02:32:36] <sustrik> so you cannot browse around much
[Friday 03 September 2010] [02:32:41] <sustrik> what you need to know:
[Friday 03 September 2010] [02:32:56] <sustrik> 1. what is it about (very roughly)
[Friday 03 September 2010] [02:33:00] <sustrik> 2. does it run on my OS
[Friday 03 September 2010] [02:33:09] <sustrik> 3. does it work with my language
[Friday 03 September 2010] [02:33:37] <sustrik> 4. i want to see a simple code example just to make sure it’s not a utter shit
[Friday 03 September 2010] [02:33:49] <sustrik> after that i want a one click download
[Friday 03 September 2010] [02:34:09] <sjampoo> All that is true, but you are missing the biggest yet.
[Friday 03 September 2010] [02:34:15] <sustrik> yes?
[Friday 03 September 2010] [02:34:16] <sjampoo> “Why should I care”
[Friday 03 September 2010] [02:34:27] <sustrik> because i have a problem to solve
[Friday 03 September 2010] [02:34:37] <sustrik> i have to deliver tomorrow
[Friday 03 September 2010] [02:34:52] <sustrik> network part is still missing :(
[Friday 03 September 2010] [02:34:53] <sustrik> etc.
[Friday 03 September 2010] [02:35:11] <sjampoo> Yes, but that isn’t explained currently on zmq.com
[Friday 03 September 2010] [02:35:27] <sustrik> it can be improved obviously
[Friday 03 September 2010] [02:35:39] <sustrik> but the main point is that most people are not browsing for fun
[Friday 03 September 2010] [02:35:44] <sustrik> they are looking for solution
[Friday 03 September 2010] [02:36:16] <sustrik> so what they ask is “would this solve my problem” rather then “what it is good for”
[Friday 03 September 2010] [02:36:24] <sjampoo> Absolutetely. Ok, I would really add a link sidebar on the left side of zeromq.com
[Friday 03 September 2010] [02:36:58] <sustrik> what links?
[Friday 03 September 2010] [02:38:16] <sjampoo> all the ‘content’ links on zeromq.org, links to Github, Blog and Mailinglist
[Friday 03 September 2010] [02:38:40] <sustrik> too confusing imo
[Friday 03 September 2010] [02:39:09] <sustrik> that’s good once you are using the thing
[Friday 03 September 2010] [02:39:34] <sustrik> the zeromq.com is there just to streamline the process up to the dowload phase
[Friday 03 September 2010] [02:40:35] <sustrik> once you’re developing with 0mq you would naturally switch to the community site
[Friday 03 September 2010] [02:41:34] * lestrrat didn’t even know there was a zeromq.com
[Friday 03 September 2010] [02:41:57] <sustrik> it’s new
[Friday 03 September 2010] [02:42:06] <sustrik> created day before yesterdat
[Friday 03 September 2010] [02:43:23] <sustrik> sjampoo: i think you are right about .org, it’s less readable now
[Friday 03 September 2010] [02:43:36] <sustrik> but .com (or however it is called) has it meaning
[Friday 03 September 2010] [02:43:57] <sustrik> althought the user it is aimed at are invisible to you – visible through google analytics though
[Friday 03 September 2010] [02:44:37] <sjampoo> sustrik: I can see what it tries to do, but I am afraid that zeromq.com in its current state will not leave the user with a ‘WOW’ feeling.
[Friday 03 September 2010] [02:44:56] <sustrik> that’s not it goal i think
[Friday 03 September 2010] [02:44:57] <sjampoo> Atleast get some sort of ‘Who is already using zeromq’
[Friday 03 September 2010] [02:45:08] <sustrik> hm
[Friday 03 September 2010] [02:45:23] <sustrik> do you look at that kind of think when inspecting unknown projects?
[Friday 03 September 2010] [02:45:35] <sjampoo> sustrik: absolutely
[Friday 03 September 2010] [02:45:42] <sjampoo> Is it worth my time to investigate
[Friday 03 September 2010] [02:45:48] <sjampoo> will it gain me anything?
[Friday 03 September 2010] [02:45:54] <sjampoo> Or is it something flunky and experimental
[Friday 03 September 2010] [02:46:06] <sustrik> it may be part of “making sure it’s not a shit” step
[Friday 03 September 2010] [02:46:17] <sjampoo> exactly
[Friday 03 September 2010] [02:47:18] <sustrik> what about the “introductions”?
[Friday 03 September 2010] [02:47:23] <sustrik> not good enough?
[Friday 03 September 2010] [02:47:55] <sjampoo> well, I was more thinking about some company profiles that are currently using zeromq
[Friday 03 September 2010] [02:48:52] <sjampoo> And it really should be put on the front page, especially if you take the (no doubt high) bounce rate into account.
[Friday 03 September 2010] [02:49:06] <sustrik> does reading a list of company names you never heard about before make you more happy then quotes?
[Friday 03 September 2010] [02:49:16] <sjampoo> hehe
[Friday 03 September 2010] [02:49:16] <sustrik> (i’m not trolling, i’m really interested)
[Friday 03 September 2010] [02:49:26] <sjampoo> sustrik: interesting question
[Friday 03 September 2010] [02:49:33] <sjampoo> no, I think both are well
[Friday 03 September 2010] [02:49:38] <sjampoo> uh i mean will do
[Friday 03 September 2010] [02:49:48] <sjampoo> And I am pretty sure that it doesn’t really matter who says what
[Friday 03 September 2010] [02:50:07] <sjampoo> that’s why astroturfing is so succesful.
[Friday 03 September 2010] [02:50:18] <sjampoo> We people are so sensitive to what others think
[Friday 03 September 2010] [02:50:46] <sustrik> yeah, but its unlikely to get madonna say anything about 0mq
[Friday 03 September 2010] [02:50:57] <sustrik> so you are still stuck with quotes from people you don’t know
[Friday 03 September 2010] [02:51:19] <sjampoo> exactly, and that doesn’t matter
[Friday 03 September 2010] [02:51:28] <sustrik> yes, quote just sounds well
[Friday 03 September 2010] [02:51:40] <sustrik> because someone invested 10 seconds to write it
[Friday 03 September 2010] [02:52:12] <sustrik> company name is less personal
[Friday 03 September 2010] [02:53:13] <sustrik> now, as for moving it to front page
[Friday 03 September 2010] [02:53:27] <sustrik> do you read endorsements first?
[Friday 03 September 2010] [02:53:47] <sustrik> even before what the project is and whether it will run with your OS/language
[Friday 03 September 2010] [02:54:07] * sustrik is doing it other way round
[Friday 03 September 2010] [02:56:12] <sjampoo> I usually read from top top bottom, but sometime I get distracted by bigger and bolder fonts.
[Friday 03 September 2010] [02:56:26] <sjampoo> and when your lucky I actually click on a link instead of back
[Friday 03 September 2010] [02:56:35] <sustrik> hm
[Friday 03 September 2010] [02:56:53] <sustrik> what i do is a type keywords into google bar
[Friday 03 September 2010] [02:57:03] <sustrik> i get a list of more or less matching projects
[Friday 03 September 2010] [02:57:13] <guido_g> re
[Friday 03 September 2010] [02:57:21] <sustrik> then i look at each one discarding those that won’t fit
[Friday 03 September 2010] [02:57:27] <sjampoo> sustrik: yes, exactly
[Friday 03 September 2010] [02:57:28] <guido_g> ahhh web site issues :)
[Friday 03 September 2010] [02:57:32] <sustrik> i spend say 30 seconds on each page
[Friday 03 September 2010] [02:57:43] <guido_g> could one please reconsider the size of the logo?
[Friday 03 September 2010] [02:57:53] <sustrik> do you want it bigger?
[Friday 03 September 2010] [02:57:55] <sustrik> :)
[Friday 03 September 2010] [02:57:59] <guido_g> when viewed on small screen its taking too much space
[Friday 03 September 2010] [02:58:09] * sustrik makes a note
[Friday 03 September 2010] [02:58:11] <guido_g> sustrik: bad boy! :)
[Friday 03 September 2010] [02:58:28] <guido_g> it’s more than 1/3 of vertical screen space
[Friday 03 September 2010] [02:58:36] <sjampoo> For a page to appear in Google it needs to be indexed with the right terms.
[Friday 03 September 2010] [02:58:38] <sustrik> what’s you screen size?
[Friday 03 September 2010] [02:58:42] <sustrik> just for reference
[Friday 03 September 2010] [02:58:57] <guido_g> and this color makes one aggressive...
[Friday 03 September 2010] [02:59:05] <guido_g> 13.6’‘
[Friday 03 September 2010] [02:59:14] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [02:59:14] <sustrik> i meant in pixels
[Friday 03 September 2010] [02:59:15] <sjampoo> guido_g: ah yes colors
[Friday 03 September 2010] [02:59:19] <sjampoo> lets discuss that next
[Friday 03 September 2010] [02:59:30] * sjampoo grins
[Friday 03 September 2010] [02:59:42] <sjampoo> I actually like the red big logo
[Friday 03 September 2010] [02:59:57] <sustrik> yes, i thought of red logo + shades of grey underneath
[Friday 03 September 2010] [03:00:09] <guido_g> 1280x800
[Friday 03 September 2010] [03:00:15] <sjampoo> it would make a nice tattoo
[Friday 03 September 2010] [03:00:15] <sustrik> it’s kind of dull, but maybe hurts eyes less
[Friday 03 September 2010] [03:00:30] <sustrik> guido_g: thanks
[Friday 03 September 2010] [03:01:06] <sustrik> let’s finish the content thing:
[Friday 03 September 2010] [03:01:08] <guido_g> it’s not that bad when using 1920x1200 on a 27’’ display
[Friday 03 September 2010] [03:01:26] <sustrik> so the front page at zeromq.com is meant to convey, in 30 secs
[Friday 03 September 2010] [03:01:34] <guido_g> but still, takes to much space that could’ve been used for better things
[Friday 03 September 2010] [03:01:39] <sustrik> whether the project is usable in your particular case or not
[Friday 03 September 2010] [03:01:54] <sustrik> guido_g: yes, we can do some experimenting with it
[Friday 03 September 2010] [03:02:37] <guido_g> if it should be for new users, why must i have cookies allowed then?
[Friday 03 September 2010] [03:02:57] <sustrik> what does it do if you don’t?
[Friday 03 September 2010] [03:03:13] <guido_g> it shows a message to activate cookies
[Friday 03 September 2010] [03:03:21] <sustrik> yuck
[Friday 03 September 2010] [03:03:33] * sustrik is not a web person, let me ask pieter
[Friday 03 September 2010] [03:03:33] <guido_g> Can’t proceed, you should accept cookies for this domain.
[Friday 03 September 2010] [03:04:23] <guido_g> i mean, i know mq, so i’ll activate cookies
[Friday 03 September 2010] [03:04:45] <guido_g> but if this would have been the first impression, i’d closed the tab
[Friday 03 September 2010] [03:05:07] <sustrik> right, i am not sure what may be the case. if it’s wikidot requiring the cookies we can move the page elsewhere
[Friday 03 September 2010] [03:05:20] <guido_g> i think it’s wikidot
[Friday 03 September 2010] [03:05:23] <sustrik> it doesn’t have to be a wiki after all
[Friday 03 September 2010] [03:05:42] <guido_g> because you need to allow cross-domain cookies (or how they’re called)
[Friday 03 September 2010] [03:06:01] <sustrik> does that happen with .org as well?
[Friday 03 September 2010] [03:06:16] <guido_g> a community site should be accessible even for users taking their privacy serious
[Friday 03 September 2010] [03:06:25] <guido_g> i’ll take a look
[Friday 03 September 2010] [03:06:46] <guido_g> yes
[Friday 03 September 2010] [03:08:10] <guido_g> btw, wikidot cookies
[Friday 03 September 2010] [03:09:39] <sustrik> hm, i switched off cookies and it still works for me
[Friday 03 September 2010] [03:09:50] <sustrik> what browser are you using?
[Friday 03 September 2010] [03:10:00] <guido_g> btw, i think for the community site we don’t need a large funny picture at the top of the entry page
[Friday 03 September 2010] [03:10:09] <guido_g> opera 10.60
[Friday 03 September 2010] [03:11:58] <sustrik> hm
[Friday 03 September 2010] [03:12:14] <guido_g> i don’t like the sound of “hm” :)
[Friday 03 September 2010] [03:12:52] <sustrik> i was just wondering if i should install opera :)
[Friday 03 September 2010] [03:13:05] <guido_g> hehe
[Friday 03 September 2010] [03:13:09] <sustrik> but let’s rather ask at wikidot
[Friday 03 September 2010] [03:13:14] * sustrik makes another note
[Friday 03 September 2010] [03:13:24] <guido_g> ok, thx for listening
[Friday 03 September 2010] [03:13:47] * guido_g is off to $day-job
[Friday 03 September 2010] [03:13:52] <sustrik> thanks for the feedback
[Friday 03 September 2010] [03:14:01] <guido_g> no problem
[Friday 03 September 2010] [03:14:13] Quit guido_g has left this server (Quit: Boom...).
[Friday 03 September 2010] [03:23:45] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 03 September 2010] [03:36:41] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 03 September 2010] [03:39:08] Quit xrfang has left this server (Remote host closed the connection).
[Friday 03 September 2010] [03:47:33] <guido_g> re
[Friday 03 September 2010] [03:52:29] Nick vic is now known as Guest26371.
[Friday 03 September 2010] [04:13:34] <sustrik> guido_g: the article at acm queue is good
[Friday 03 September 2010] [04:13:45] <sustrik> i wanted to write something like this for a long time
[Friday 03 September 2010] [04:13:53] <sustrik> but never had enough time to actually do it
[Friday 03 September 2010] [04:14:25] <guido_g> oh, so another dream died :)
[Friday 03 September 2010] [04:14:30] <
halfie_> hi, can I modify the publish-subscribe model so that the “queue” is read-only?
[Friday 03 September 2010] [04:14:32] <
halfie_> I don
[Friday 03 September 2010] [04:14:53] <guido_g> huh?
[Friday 03 September 2010] [04:14:57] <guido_g> what does that mean?
[Friday 03 September 2010] [04:15:04] <guido_g> SUB sockets are read only...
[Friday 03 September 2010] [04:15:41] <
halfie_> guido_g, I mean that the subscriber shouldn’t remove the message
[Friday 03 September 2010] [04:15:57] <sustrik> you need a database, not a messaging system
[Friday 03 September 2010] [04:16:18] <sustrik> publisher -> INSERT
[Friday 03 September 2010] [04:16:19] <guido_g> ack
[Friday 03 September 2010] [04:16:23] <sustrik> subscriber -> SELECT
[Friday 03 September 2010] [04:17:01] <
halfie_> I see, makes sense.
[Friday 03 September 2010] [04:17:31] <
halfie_> some kind of in-memory database with “queue” semantics would solve the problem
[Friday 03 September 2010] [04:18:11] <guido_g> berkeley db
[Friday 03 September 2010] [04:19:12] <mato> ‘morning
[Friday 03 September 2010] [04:19:50] <
halfie_> guido_g, thanks, I will give it a shot. sustrik: thanks for clearing the situation :)
[Friday 03 September 2010] [04:20:20] <sustrik> :)
[Friday 03 September 2010] [04:20:27] <sustrik> mato: morning
[Friday 03 September 2010] [04:20:34] <sustrik> a question
[Friday 03 September 2010] [04:20:38] <mato> yup?
[Friday 03 September 2010] [04:20:39] <sustrik> a signal question!
[Friday 03 September 2010] [04:20:40] <guido_g> hi mato
[Friday 03 September 2010] [04:20:45] <mato> hi guido
[Friday 03 September 2010] [04:20:51] <sustrik> imagine this:
[Friday 03 September 2010] [04:21:03] <sustrik> binding calls 0mq’s blocking function
[Friday 03 September 2010] [04:21:20] <sustrik> 99% of the time is spent in OS’s blocking call
[Friday 03 September 2010] [04:21:32] <sustrik> 1% of time some 0mq processing is being done
[Friday 03 September 2010] [04:21:46] <sustrik> what if user Ctrl+C’s during the 1%?
[Friday 03 September 2010] [04:21:55] <mato> too bad
[Friday 03 September 2010] [04:21:58] <sustrik> :)
[Friday 03 September 2010] [04:22:02] <mato> it’ll get delivered late
[Friday 03 September 2010] [04:22:06] <mato> nothing you can do about that
[Friday 03 September 2010] [04:22:23] <mato> well, there are various weird hacks you can do about that, but believe me, you don’t want to go there
[Friday 03 September 2010] [04:22:29] <sustrik> it won’t be delivered at all no?
[Friday 03 September 2010] [04:22:50] <sustrik> the handler runs, sets a bit
[Friday 03 September 2010] [04:23:02] <sustrik> there’s no blocking call for OS to interrupt
[Friday 03 September 2010] [04:23:06] <sustrik> so nothing happens
[Friday 03 September 2010] [04:23:11] <mato> hmm
[Friday 03 September 2010] [04:23:16] <sustrik> 0mq then calls a blocking call
[Friday 03 September 2010] [04:23:20] <sustrik> and freezes
[Friday 03 September 2010] [04:23:47] <mato> oh dear
[Friday 03 September 2010] [04:23:50] <mato> yes, you may be right
[Friday 03 September 2010] [04:24:02] <sustrik> the whole EINTR thing seems broken
[Friday 03 September 2010] [04:24:06] <mato> i have always said signal handling is broken
[Friday 03 September 2010] [04:24:10] <mato> to put it very politely
[Friday 03 September 2010] [04:24:16] <sustrik> is there any way to make it work?
[Friday 03 September 2010] [04:24:31] <mato> hmm, well, maybe
[Friday 03 September 2010] [04:24:42] <sustrik> imagine you are omnipotent
[Friday 03 September 2010] [04:24:44] <mato> but it’s so horribly racy i wouldn’t want to go there
[Friday 03 September 2010] [04:24:56] <mato> oh, if i’m omnipotent then 0mq is in kernel space, problem goes away
[Friday 03 September 2010] [04:24:57] <sustrik> you can change the language runtimes etc.
[Friday 03 September 2010] [04:25:08] <mato> ah, less than god (linus), ok...
[Friday 03 September 2010] [04:25:11] <mato> just a sub-god :)
[Friday 03 September 2010] [04:25:22] <sustrik> what i’m asking: is there a canonical way to use EINTR
[Friday 03 September 2010] [04:25:28] <sustrik> something that works?
[Friday 03 September 2010] [04:25:31] <mato> well, there is a canonical way i would do it
[Friday 03 September 2010] [04:25:40] <mato> this is one of the ideas i had yesterday
[Friday 03 September 2010] [04:25:44] <mato> it might not be right
[Friday 03 September 2010] [04:25:48] * sustrik is listening
[Friday 03 September 2010] [04:25:50] <mato> but assuming you can hack the language runtime
[Friday 03 September 2010] [04:26:36] <mato> then what you need to do is chuck a call to a thread-safe “zmq_interrupt()” into the low-level signal handler in the runtime
[Friday 03 September 2010] [04:26:41] <mato> i.e. the bit that “sets a flag”
[Friday 03 September 2010] [04:26:58] <mato> this zmq_interrupt() would cause all application API calls to immediately return with EINTR
[Friday 03 September 2010] [04:27:13] <mato> it’s ugly, but it should work
[Friday 03 September 2010] [04:27:37] <sustrik> tweaking it a bit you end up with chained signal handlers
[Friday 03 September 2010] [04:27:49] <mato> yes, except there’s no API for that
[Friday 03 September 2010] [04:27:53] <sustrik> exactly
[Friday 03 September 2010] [04:27:59] <mato> so you don’t want to go there :)
[Friday 03 September 2010] [04:28:08] <mato> trying to emulate chained signal handlers is madness
[Friday 03 September 2010] [04:28:16] <sustrik> so the whole system is really broken
[Friday 03 September 2010] [04:28:20] <mato> yes
[Friday 03 September 2010] [04:28:46] <sustrik> ok, what i do is froward the EINTR to the binding
[Friday 03 September 2010] [04:28:49] <mato> it doesn’t account for the fact that more than one “entity” inside the application might want to know if a signal arrived
[Friday 03 September 2010] [04:28:54] <sustrik> that would work in99% of cases
[Friday 03 September 2010] [04:29:10] <sustrik> the rest...
[Friday 03 September 2010] [04:29:11] <mato> “forward the EINTR to the binding”? how?
[Friday 03 September 2010] [04:29:11] <sustrik> shrug
[Friday 03 September 2010] [04:29:25] <sustrik> if (errno == EINTR) return -1;
[Friday 03 September 2010] [04:29:30] <mato> oh, that, yes...
[Friday 03 September 2010] [04:29:34] <mato> hmm
[Friday 03 September 2010] [04:29:39] <mato> let me check something
[Friday 03 September 2010] [04:31:34] Join plq has joined this channel (
~plq@78.189.30.218).
[Friday 03 September 2010] [04:34:32] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 03 September 2010] [04:44:26] <mato> sustrik: ok, so, yes, forwarding EINTR is the only way at the moment
[Friday 03 September 2010] [04:44:33] <sustrik> ack
[Friday 03 September 2010] [04:44:41] <sustrik> well, there’s another wat
[Friday 03 September 2010] [04:44:43] <sustrik> way
[Friday 03 September 2010] [04:44:45] <mato> yes?
[Friday 03 September 2010] [04:45:03] <sustrik> on each entry to potentially blocking 0mq function
[Friday 03 September 2010] [04:45:11] <sustrik> store the old signal handler
[Friday 03 September 2010] [04:45:14] <sustrik> install your own
[Friday 03 September 2010] [04:45:15] <sustrik> etc.
[Friday 03 September 2010] [04:45:25] <sustrik> then on exit return it to the original state
[Friday 03 September 2010] [04:45:27] <mato> that’s the naive way, yes
[Friday 03 September 2010] [04:45:29] <mato> except
[Friday 03 September 2010] [04:45:35] <mato> IT WILL NEVER WORK :)
[Friday 03 September 2010] [04:45:41] <sustrik> why so?
[Friday 03 September 2010] [04:46:09] <mato> because, and this is what i just double-checked, all the POSIX docs explicitly say “never, ever, ever, try to mix different signal handling mechanisms and threads”
[Friday 03 September 2010] [04:46:25] <mato> phone, hang on
[Friday 03 September 2010] [04:46:56] <sustrik> not sure what that means, but ok, i don’t want to implement that kind of hack anyway
[Friday 03 September 2010] [04:48:28] <mato> let me explain
[Friday 03 September 2010] [04:48:40] <mato> the problem is it only works if you have 1 thread
[Friday 03 September 2010] [04:48:41] <mato> 1 OS thread
[Friday 03 September 2010] [04:48:45] <mato> and no more
[Friday 03 September 2010] [04:49:18] <mato> if you start doing that kind of stuff it will break any application that might actually trying to be do things the right way in a MT envinronment, which is using a sigwait() thread
[Friday 03 September 2010] [04:49:29] <mato> you have no way of knowing what the application is doing...
[Friday 03 September 2010] [04:49:34] <mato> does that make sense?
[Friday 03 September 2010] [04:50:00] <sustrik> i have no idea how sigwait style interacts with the classic style
[Friday 03 September 2010] [04:50:13] <mato> badly
[Friday 03 September 2010] [04:50:16] <mato> i.e. never do that
[Friday 03 September 2010] [04:50:16] <sustrik> but i don’t really care
[Friday 03 September 2010] [04:50:30] <sustrik> so for now i’ll return EINTR
[Friday 03 September 2010] [04:50:31] <mato> yeah, but if you start mucking with signal handlers then you start caring :)
[Friday 03 September 2010] [04:50:33] <sustrik> when available
[Friday 03 September 2010] [04:50:37] <mato> yes
[Friday 03 September 2010] [04:50:49] <mato> returning EINTR will solve the problem “most of the time”
[Friday 03 September 2010] [04:50:50] <sustrik> so sometimes Ctrl+C won’t work
[Friday 03 September 2010] [04:50:53] Quit jsimmons has left this server (Quit: Leaving).
[Friday 03 September 2010] [04:50:58] <mato> but you can still get signal loss someimes
[Friday 03 September 2010] [04:51:01] <sustrik> however, user can press Ctrl+C again
[Friday 03 September 2010] [04:51:11] <mato> this just needs to be documented, that’s all
[Friday 03 September 2010] [04:51:35] * mato sees the 0MQ FAQ on signal handling becoming into a treatise on the brokenness of signals :)
[Friday 03 September 2010] [04:52:03] <sustrik> yeah
[Friday 03 September 2010] [04:52:11] <mato> sustrik: anyway, can i suggest the following process for this:
[Friday 03 September 2010] [04:52:28] <mato> sustrik: 1. Test the solution, as discussed yesterday.
[Friday 03 September 2010] [04:52:43] <mato> sustrik: 2. Post a patch for review, along with your “I’m (almost) an idiot, sorry”
[Friday 03 September 2010] [04:53:19] <mato> sustrik: 3. I will reply to that describing all I know about the problem so that the binding people can understand what’s going on (since they’re the most affected)
[Friday 03 September 2010] [04:53:56] <mato> sustrik: 4. Some consensus will be reached on whether or not this goes into 2.0.x (I’m not opposed either way, it’s a fix for breakage on our part)
[Friday 03 September 2010] [04:54:30] <mato> sustrik: 5. We may learn something useful from the discussion, and I can also bring up the zmq_interrupt() idea which might give a solution for those language runtimes that want to solve this 100%...
[Friday 03 September 2010] [04:54:41] <mato> sustrik: what do you think?
[Friday 03 September 2010] [04:56:31] <mato> phone call, back in 20mins
[Friday 03 September 2010] [04:56:33] <pieterh> re
[Friday 03 September 2010] [04:56:40] <pieterh> i lost my phone and have been offline
[Friday 03 September 2010] [04:56:42] <pieterh> :-(
[Friday 03 September 2010] [04:58:02] <pieterh> sjampoo: i read your remarks about the website
[Friday 03 September 2010] [04:58:11] <pieterh> they make sense and we’ll evolve this over time
[Friday 03 September 2010] [04:59:50] Quit
gebi_ has left this server (Ping timeout: 272 seconds).
[Friday 03 September 2010] [05:20:39] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 03 September 2010] [05:23:29] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [05:30:36] Quit poswald has left this server (Quit: poswald).
[Friday 03 September 2010] [05:35:23] Quit poswald has left this server (Ping timeout: 260 seconds).
[Friday 03 September 2010] [05:46:21] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Friday 03 September 2010] [05:54:53] <mato> sustrik: re
[Friday 03 September 2010] [05:54:59] <mato> sustrik: are you happy with what i propose?
[Friday 03 September 2010] [05:55:15] <mato> sustrik: it seems like the best way to move forward on the signal handling issues
[Friday 03 September 2010] [07:00:57] <sustrik> mato: yes
[Friday 03 September 2010] [07:01:00] <sustrik> re
[Friday 03 September 2010] [07:01:04] <mato> sustrik: see my long email
[Friday 03 September 2010] [07:01:13] <mato> sustrik: i’ve tried to explain the issue, complex as it is
[Friday 03 September 2010] [07:01:27] <sustrik> ok, i’m on it
[Friday 03 September 2010] [07:08:05] Quit guido_g has left this server (Quit: Boom...).
[Friday 03 September 2010] [07:16:43] Join pieterh has joined this channel (
~ph@217.75.64.166).
[Friday 03 September 2010] [07:30:29] <mato> ok, another big long email sent re the Python GIL stuff
[Friday 03 September 2010] [07:30:44] <mato> somewhat braindump format, hopefully it’s clear enough
[Friday 03 September 2010] [08:12:23] <arnorhs> yo
[Friday 03 September 2010] [08:12:42] <arnorhs> Which programming language has the best zero mq bindings?
[Friday 03 September 2010] [08:12:54] <arnorhs> Which one is the most used / most stable?
[Friday 03 September 2010] [08:15:53] <guido_g> mq itself is programmin in c++, so this should work
[Friday 03 September 2010] [08:16:08] <guido_g> c is delivered with the core, so does work too
[Friday 03 September 2010] [08:16:38] <sustrik> yes, all the bindings use C API underneath
[Friday 03 September 2010] [08:16:42] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [08:17:04] <guido_g> the other bindings are quite stable, if you don’t insinst on using bleeding edge versions
[Friday 03 September 2010] [08:26:55] <arnorhs> ok, thanks
[Friday 03 September 2010] [08:27:03] <arnorhs> I will go for fortran then
[Friday 03 September 2010] [08:27:11] <arnorhs> :)
[Friday 03 September 2010] [08:27:24] <arnorhs> I’m guessing the python bindings are fine?
[Friday 03 September 2010] [08:30:17] <sustrik> yes, i think they are most stable among the whole bunch
[Friday 03 September 2010] [08:51:24] <arnorhs> ok, thanks
[Friday 03 September 2010] [08:53:11] <gebi> are there any plans to add {active, once} mode to the erlang bindings?
[Friday 03 September 2010] [08:53:35] Quit arnorhs has left this server (Remote host closed the connection).
[Friday 03 September 2010] [08:58:54] <sustrik> gebi: what’s once?
[Friday 03 September 2010] [09:23:46] Quit plq has left this server (Remote host closed the connection).
[Friday 03 September 2010] [09:30:37] <gebi> sustrik: only deliver ONE message
[Friday 03 September 2010] [09:31:07] <gebi> with active,true is unuseable because it will fill up your process message queue and make it unusable
[Friday 03 September 2010] [09:31:45] <gebi> active,false is also not ideal because processes block and do not process other messages
[Friday 03 September 2010] [10:31:44] <sustrik> gebi: how does that work
[Friday 03 September 2010] [10:31:50] <sustrik> ok, it delivers one message
[Friday 03 September 2010] [10:31:55] <sustrik> when then?
[Friday 03 September 2010] [10:32:10] <sustrik> you still have to have a way to get next message
[Friday 03 September 2010] [10:32:20] <sustrik> mato: are you here?
[Friday 03 September 2010] [10:35:26] <sustrik> gebi: does it mean there’s at most one message in erlang queue?
[Friday 03 September 2010] [10:35:42] <sustrik> if so, how do you tell erlang to behave that way?
[Friday 03 September 2010] [10:59:09] <gebi> sustrik: it should be a property of the erlang 0mq binding
[Friday 03 September 2010] [11:00:20] <sustrik> :)
[Friday 03 September 2010] [11:00:37] <sustrik> i know, i was just interested whether erlnag as such has such feature
[Friday 03 September 2010] [11:00:41] <gebi> with active, once it just sends delivers a single received datagram into the message queue and stops then
[Friday 03 September 2010] [11:00:42] <sustrik> erlang*
[Friday 03 September 2010] [11:01:03] <sustrik> that’s erlang’s native networking API?
[Friday 03 September 2010] [11:01:49] <gebi> yea it’s a property of gen_tcp
[Friday 03 September 2010] [11:02:16] * sustrik is checking it
[Friday 03 September 2010] [11:03:13] <gebi> you just call inet:setopts(S,[{active,once}]) when you want to get another datagram delivered into your messagequeue
[Friday 03 September 2010] [11:04:25] <sustrik> aha
[Friday 03 September 2010] [11:04:33] <sustrik> how is that different from passive?
[Friday 03 September 2010] [11:04:55] <gebi> what do you mean with passive?
[Friday 03 September 2010] [11:05:11] <sustrik> active=false
[Friday 03 September 2010] [11:05:23] Quit dos000 has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [11:05:23] Quit sachinr has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [11:06:34] <gebi> with active false you have an effectively dead process (all your receiving processes hang on erlzmq:receive)
[Friday 03 September 2010] [11:07:36] <gebi> with all the bad sideeffects
[Friday 03 September 2010] [11:08:20] <sustrik> ah, it blocks threads in erlnag thread pool?
[Friday 03 September 2010] [11:08:25] <sustrik> erlnag*
[Friday 03 September 2010] [11:08:28] <sustrik> erlang*
[Friday 03 September 2010] [11:08:34] <gebi> no erlang processes
[Friday 03 September 2010] [11:09:15] <sustrik> does it matter? if process is up to receiving a message am there’s no message available it hangs anyway
[Friday 03 September 2010] [11:09:55] <gebi> if it “hangs” on receiving a erlang message it is alive, if it hangs on receiving a message from the 0mq binding it is dead
[Friday 03 September 2010] [11:10:23] <sustrik> does “dead” mean that it is ubable to process erlang messages?
[Friday 03 September 2010] [11:10:25] <gebi> meaning it wont process erlang messages till at least _one_ 0mq message gets delivered (and erlzmq:recv returns)
[Friday 03 September 2010] [11:10:32] <sustrik> ok, got it
[Friday 03 September 2010] [11:10:48] <sustrik> makes sense
[Friday 03 September 2010] [11:10:51] <gebi> this _is_ the fastest mode, though most of the time not desireable
[Friday 03 September 2010] [11:11:21] <sustrik> good, i’ll probably have a look at it next week
[Friday 03 September 2010] [11:11:26] <gebi> same with active,true, processing the message-queue in erlang is NOT O(1)
[Friday 03 September 2010] [11:11:39] <sustrik> ack
[Friday 03 September 2010] [11:11:57] <gebi> so it’s kinda ugly to have a big messagequeue, thus active,true is not possibel without another protocol above for flowcontrole
[Friday 03 September 2010] [11:12:00] <gebi> thx :)
[Friday 03 September 2010] [11:12:37] <sustrik> i was actually wondering how the flow control thing can be handled in erlang
[Friday 03 September 2010] [11:12:48] <sustrik> you’ve gave me the answer for free :)
[Friday 03 September 2010] [11:12:53] <gebi> heh, np :)
[Friday 03 September 2010] [11:31:58] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r9e7a9d1 10/ src/zmq.cpp : zmq_poll returns EINTR if signal interrupts it -
http://bit.ly/cWOXej[Friday 03 September 2010] [11:31:59] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * rb059116 10/ (9 files): EINTR is propogated from signaler_t::recv all the way up to the 0MQ API -
http://bit.ly/bOz5Ui[Friday 03 September 2010] [11:32:01] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r63706a9 10/ (doc/zmq_poll.txt doc/zmq_recv.txt doc/zmq_send.txt): documentation modified to reflect the EINTR change -
http://bit.ly/92QIrS[Friday 03 September 2010] [11:51:44] <cremes> sustrik: i get an assertion failure when running the EINTR branch
[Friday 03 September 2010] [11:55:25] <sustrik> cremes: let me see
[Friday 03 September 2010] [12:01:47] Quit
feroz_ has left this server (Quit:
feroz_).
[Friday 03 September 2010] [12:07:10] <sustrik> cremes: try now
[Friday 03 September 2010] [12:08:03] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r1bc4e9e 10/ src/app_thread.cpp : one more EINTR fix -
http://bit.ly/ciucLd[Friday 03 September 2010] [12:10:07] <cremes> sustrik: works
[Friday 03 September 2010] [12:11:06] <sustrik> you mean Ctrl+C stops the app?
[Friday 03 September 2010] [12:11:12] <cremes> yessir
[Friday 03 September 2010] [12:11:15] <sustrik> woo-hoo!
[Friday 03 September 2010] [12:11:19] <cremes> indeed!
[Friday 03 September 2010] [12:11:30] <sustrik> that’s a bug ~2 years old
[Friday 03 September 2010] [12:11:43] <sustrik> maybe 2 1/2
[Friday 03 September 2010] [12:11:58] <sustrik> until now nobody had any clue how to solve ti
[Friday 03 September 2010] [12:12:01] <cremes> wow, good catch
[Friday 03 September 2010] [12:12:02] <sustrik> great!
[Friday 03 September 2010] [12:12:26] <cremes> is this going into 2.0.9 or 2.1?
[Friday 03 September 2010] [12:12:30] <sustrik> let’s wait for bgranger and lestrrat to confirm it’s working first
[Friday 03 September 2010] [12:12:38] <cremes> sure thing
[Friday 03 September 2010] [12:12:46] <sustrik> i am afraid of pushing it to a stable version
[Friday 03 September 2010] [12:12:51] <cremes> i’ll drop a note to the ML to make my report “official”
[Friday 03 September 2010] [12:13:07] <sustrik> yes, good idea
[Friday 03 September 2010] [12:13:22] <cremes> yeah, i wrote to the ML with my reasoning for this change to go into 2.1 but not 2.0.x
[Friday 03 September 2010] [12:13:30] <sustrik> +1
[Friday 03 September 2010] [12:13:51] <sustrik> at least it’s a motivation for migration to 2.1 :)
[Friday 03 September 2010] [12:16:17] <cremes> oh yeah, i’m looking forward to the thread migration changes too
[Friday 03 September 2010] [12:16:31] <cremes> and the new FD_EVENT stuff
[Friday 03 September 2010] [12:18:49] <sustrik> it’s there to test already
[Friday 03 September 2010] [12:19:00] <sustrik> it may be a bit buggy
[Friday 03 September 2010] [12:19:02] Quit
dos000_ has left this server (Quit: Leaving).
[Friday 03 September 2010] [12:19:03] <sustrik> but still...
[Friday 03 September 2010] [12:19:42] <cremes> sustrik: i’m pretty busy building out a production app using 0mq 2.0.x so i haven’t had any time to test 2.1.x
[Friday 03 September 2010] [12:20:00] <cremes> when you get closer to a release, you should ping the bindings authors and ask us to test
[Friday 03 September 2010] [12:20:06] <sustrik> sure, just updating you on the status
[Friday 03 September 2010] [12:20:12] <cremes> cool, i appreciate it
[Friday 03 September 2010] [12:20:14] <sustrik> we’ll do some testing ourselves
[Friday 03 September 2010] [12:20:19] <sustrik> in the next few weeks
[Friday 03 September 2010] [12:20:28] <sustrik> then it’s binding autohr’s turn
[Friday 03 September 2010] [12:20:45] <cremes> do you have a time frame for the 2.1 release? 2-4 weeks? 4-6 weeks?
[Friday 03 September 2010] [12:22:09] <sustrik> hm, it’s up to mato, he’s going to do the testing
[Friday 03 September 2010] [12:22:30] <sustrik> bet yeah, something between 2 ans 6 weeks i would say
[Friday 03 September 2010] [12:27:05] <cremes> ok
[Friday 03 September 2010] [12:28:10] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [13:14:25] Quit xla has left this server (Quit: leaving).
[Friday 03 September 2010] [13:31:21] <ModusPwnens> So, I updated to 2.0.8 and I still have the problem of the benchmarking programs spitting out errors if you enter in very large parameters
[Friday 03 September 2010] [13:31:23] Quit gebi has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [13:32:07] <ModusPwnens> i’m using windows 7 on both operating systems, but the system running the remote end is 64-bit
[Friday 03 September 2010] [13:32:37] <ModusPwnens> and here is the pastebin with the error emssage and input
[Friday 03 September 2010] [13:35:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [13:40:34] <sustrik> ModusPwnens: throughput test?
[Friday 03 September 2010] [13:40:46] <sustrik> try increasing the sleep period in remote_the
[Friday 03 September 2010] [13:41:02] <sustrik> otherwise it doesn’t have enough time to send all the messages
[Friday 03 September 2010] [13:41:57] <ModusPwnens> Ok, I will try that sustrik
[Friday 03 September 2010] [13:42:08] <sustrik> cremes: nice!
[Friday 03 September 2010] [13:42:24] <sustrik> ha, that’s the guy complaining about the website!
[Friday 03 September 2010] [13:42:27] <sustrik> :)
[Friday 03 September 2010] [13:42:38] <cremes> it’s always good to see people writing articles on our favorite networking library
[Friday 03 September 2010] [13:43:23] <sustrik> yes, given how dull topic the networking is, it’s interesting that people even bother
[Friday 03 September 2010] [13:43:40] <cremes> heh
[Friday 03 September 2010] [13:43:42] <sustrik> btw, the site is back to normal
[Friday 03 September 2010] [13:45:51] <cremes> i see that; btw, i do like the new
*.com site
[Friday 03 September 2010] [13:45:54] <cremes> nice and simple
[Friday 03 September 2010] [13:46:02] Part mikejs has left this channel.
[Friday 03 September 2010] [13:46:19] <sustrik> few minor things to fix still...
[Friday 03 September 2010] [13:46:40] <sustrik> the link in the bottom look like there were 6 of them
[Friday 03 September 2010] [13:46:44] <sustrik> while there are only 3
[Friday 03 September 2010] [13:48:15] <sustrik> etc.
[Friday 03 September 2010] [14:06:46] <ModusPwnens> sustrik: I tried your suggestion and received a different error message
[Friday 03 September 2010] [14:07:21] <sustrik> what error message?
[Friday 03 September 2010] [14:07:51] <ModusPwnens> I just added it to the bottom
[Friday 03 September 2010] [14:08:04] <sustrik> well, not enough memory
[Friday 03 September 2010] [14:08:33] <ModusPwnens> so is that expected then?
[Friday 03 September 2010] [14:08:44] <sustrik> messages are held in memory
[Friday 03 September 2010] [14:09:02] <sustrik> so if you create more than your memory can hold, it’s expected
[Friday 03 September 2010] [14:09:11] <sustrik> what you can do though
[Friday 03 September 2010] [14:09:14] <ModusPwnens> Hmm, that makes sense.
[Friday 03 September 2010] [14:09:19] <sustrik> is to use ZMQ_SWAP socket option
[Friday 03 September 2010] [14:09:28] <sustrik> to offload excessive messages to the disk
[Friday 03 September 2010] [14:10:13] <ModusPwnens> Ok. I will try that.
[Friday 03 September 2010] [14:52:54] <lvh> hey
[Friday 03 September 2010] [14:53:11] <lvh> is there any progress or change regarding the zeromq-exposing-a-fd idea?
[Friday 03 September 2010] [14:53:24] <lvh> AFAIK this is what gevent and twisted are waiting for so we can integrate zeromq.
[Friday 03 September 2010] [14:53:43] <lvh> (I want to try zeromq, but if it won’t work with all of my existing event loop-based stuff, like twisted, well, I basically can’t use it.)
[Friday 03 September 2010] [14:54:39] <sustrik> lvh: it’s in trunk
[Friday 03 September 2010] [14:55:04] <sustrik> it’s a development version so it can be a bit unstable still
[Friday 03 September 2010] [14:55:25] <lvh> sustrik: oh cool, that’s interesting
[Friday 03 September 2010] [14:55:26] <sustrik> but it has ZMQ_FD socket option to get an underlying file descriptor from a 0MQ socket
[Friday 03 September 2010] [14:55:56] <sustrik> and ZMQ_EVENTS option that allows you to check whether socket is ready for readin and/or writing
[Friday 03 September 2010] [14:55:58] <lvh> git://github.com/zeromq/zeromq2.git ?
[Friday 03 September 2010] [14:56:04] <sustrik> yes
[Friday 03 September 2010] [14:56:07] <sustrik> “master”
[Friday 03 September 2010] [14:56:28] <lvh> Wait, why do people need to know the underlying fd
[Friday 03 September 2010] [14:56:46] <sustrik> to poll on it
[Friday 03 September 2010] [14:56:49] <lvh> I thought a zmq socket abstracted away from that and not necessarily have an fd, or not necessarily have exactly one
[Friday 03 September 2010] [14:57:00] <lvh> I’m not sure I understand. Isn’t that zmq’s job still?
[Friday 03 September 2010] [14:57:11] <sustrik> how would you poll on something that’s not a fd?
[Friday 03 September 2010] [14:57:16] <lvh> Is the idea to not use the IO parts in ZMQ?
[Friday 03 September 2010] [14:57:48] <lvh> sustrik: Oh, wait, sorry, I misunderstood what it does.
[Friday 03 September 2010] [14:57:51] <sustrik> the underlying fd is not the networking fd
[Friday 03 September 2010] [14:57:55] <lvh> Right.
[Friday 03 September 2010] [14:57:58] <sustrik> it’s 0MQ’s internal fd
[Friday 03 September 2010] [14:58:05] <lvh> Right, yes, that makes a lot more sense.
[Friday 03 September 2010] [14:58:05] <sustrik> used to communicate with I/O threads
[Friday 03 September 2010] [14:58:48] * sustrik is heading for pub
[Friday 03 September 2010] [15:00:03] <lvh> sustrik: have fun
[Friday 03 September 2010] [15:05:14] Quit lvh has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [15:08:21] Join mikejs has joined this channel (
~me@mikej.st).
[Friday 03 September 2010] [15:20:42] <bgranger> foo = Float(0.00001, config=True)
[Friday 03 September 2010] [15:20:56] Quit keffo has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [15:51:31] Quit GeekGod has left this server (Quit: GeekGod).
[Friday 03 September 2010] [16:32:08] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [16:48:25] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [16:49:07] Quit
mw__ has left this server (Changing host).
[Friday 03 September 2010] [17:34:54] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 03 September 2010] [17:41:26] Quit gebi has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [17:51:13] <troutwine> Does anyone have examples of Google’s Protocol Buffers being used as a serialization format with zeromq? I’ve seen it referenced but cannot find any examples.
[Friday 03 September 2010] [17:59:28] Quit keffo has left this server (Ping timeout: 272 seconds).
[Friday 03 September 2010] [18:06:40] Quit troutwine has left this server (Quit: leaving).
[Friday 03 September 2010] [18:14:11] <erickt> anyone who works on pyzmq online today? I’m having trouble trying to make a fedora rpm for the head of the tree
[Friday 03 September 2010] [18:15:15] <erickt> the commit 2a192b9 seems to have caused fedora 13’s rpmbuild to error out
[Friday 03 September 2010] [18:21:57] Quit sjampoo has left this server (Quit: zzz).
[Friday 03 September 2010] [18:22:32] Quit sjampoo has left this server (Client Quit).
[Friday 03 September 2010] [18:43:55] <bgranger> erickt: Hi I have a few minutes...
[Friday 03 September 2010] [18:44:25] <erickt> I think I found the problem, the enums for the devices is wrong
[Friday 03 September 2010] [18:44:30] <erickt> if I understand cython
[Friday 03 September 2010] [18:44:43] <bgranger> Possibly...
[Friday 03 September 2010] [18:45:11] <bgranger> What do you think the issue it.
[Friday 03 September 2010] [18:45:12] <erickt> ZMQ_QUEUE is defined to 1 in 2.0.8, ZMQ_FORWARDER 2, ZMQ_STREAMER to 3
[Friday 03 September 2010] [18:45:37] <erickt> well i haven’t completely confirmed it, but it seems to cause rpmbuild to fail on f13 I think
[Friday 03 September 2010] [18:45:44] <bgranger> Is that different than 2.0.7?
[Friday 03 September 2010] [18:45:58] Quit __moore__ has left this server (Quit: Leaving).
[Friday 03 September 2010] [18:46:16] <bgranger> Do you have an error msg or traceback?
[Friday 03 September 2010] [18:46:29] <Tasser> what’s a context exaclty?
[Friday 03 September 2010] [18:46:54] <erickt> bgranger: no, that’s part of the problem
[Friday 03 September 2010] [18:47:22] <erickt> it’s dying somewhere deep in a redhat command that doesn’t say really what’s going on
[Friday 03 September 2010] [18:47:31] <bgranger> Ahh. It is possible that the _zmq.c needs to be re-created for 2.0.8, but I would be a bit surprised.
[Friday 03 September 2010] [18:47:35] <bgranger> Can you build by hand?
[Friday 03 September 2010] [18:47:46] <erickt> i just know that in 1770dbb I can build an rpm, but with 2a192b96 i cannot
[Friday 03 September 2010] [18:47:47] <bgranger> That should show any issues outright
[Friday 03 September 2010] [18:47:57] <erickt> yeah it builds fine, and seems to be usable
[Friday 03 September 2010] [18:48:03] <bgranger> Are you buildling against 2.0.8 of zeromq
[Friday 03 September 2010] [18:48:05] <erickt> which is why I’m confused
[Friday 03 September 2010] [18:48:06] <erickt> yeah
[Friday 03 September 2010] [18:48:24] <bgranger> Hmmm, that is odd then. Do you have the script that rpmbuild is using?
[Friday 03 September 2010] [18:48:31] <erickt> Tasser: it’s an object to hold all the threads and underlying bits that zmq needs
[Friday 03 September 2010] [18:48:38] <ModusPwnens> troutwine, are you still here? I can help you!
[Friday 03 September 2010] [18:48:44] <bgranger> I do have to leave in a few minutes, so maybe post to the list if you can’t figure it out.
[Friday 03 September 2010] [18:49:15] <ModusPwnens> Also, I am receiving more errors with the benchmarking utility
[Friday 03 September 2010] [18:49:20] <ModusPwnens> this time not associated with large parameters
[Friday 03 September 2010] [18:49:22] <Tasser> erickt, can you have more than once context? not likely?
[Friday 03 September 2010] [18:49:34] <bgranger> Tasser: multiple contexts are just fine.
[Friday 03 September 2010] [18:49:53] <Tasser> hmm
[Friday 03 September 2010] [18:49:59] <bgranger> Tasser: the general guideline is 1 per library. You can use it to isolate your own zmq stuff from that of other libs.
[Friday 03 September 2010] [18:50:00] <Tasser> I’ll have to play around a bit
[Friday 03 September 2010] [18:50:08] <erickt> that’s the rpm spec and a driver script
[Friday 03 September 2010] [18:51:55] <bgranger> Do you know how far it gets?
[Friday 03 September 2010] [18:52:21] <erickt> it compiles and it’s just determining metadata for the rpm
[Friday 03 September 2010] [18:52:24] <bgranger> erickt: sorry I need to run now, can you email the list. Also, have you talked to T. Spura about this?
[Friday 03 September 2010] [18:52:32] <erickt> nope, just found this
[Friday 03 September 2010] [18:57:32] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [19:02:32] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [19:08:28] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [19:09:07] Quit
mw__ has left this server (Changing host).
[Friday 03 September 2010] [19:40:50] Quit
feroz_ has left this server (Ping timeout: 264 seconds).
[Friday 03 September 2010] [19:41:34] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Friday 03 September 2010] [19:43:15] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [19:57:13] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [19:57:45] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [20:14:13] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [21:18:32] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [21:19:30] <lestrrat> sustrik: I’m going to be out for the day, so won’t be able to check for another 8hrs or so
[Friday 03 September 2010] [21:19:41] <lestrrat> but will do when I get back
[Friday 03 September 2010] [21:20:28] Nick
AndrewBC_ is now known as AndrewBC.
[Friday 03 September 2010] [21:22:18] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [21:46:28] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [22:09:41] Quit jsimmons has left this server (Ping timeout: 255 seconds).
[Friday 03 September 2010] [22:46:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [23:36:02] Quit bgranger has left this server (Quit: bgranger).
[Friday 03 September 2010] [23:43:39] Quit bgranger has left this server (Client Quit).
[Saturday 04 September 2010] [00:03:29] Quit keffo has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 4 01:30:09 2010
[Saturday 04 September 2010] [01:30:09] Topic The channel topic is “Welcome!”.
[Saturday 04 September 2010] [01:30:15] Mode Channel modes: no messages from outside, topic protection
[Saturday 04 September 2010] [01:30:16] Created This channel was created on 2010-02-05 10:44.
[Saturday 04 September 2010] [02:01:47] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [02:03:12] Join joevandyk has joined this channel (
~joe@98.117.99.141).
[Saturday 04 September 2010] [02:03:38] <joevandyk> hello!
[Saturday 04 September 2010] [02:04:26] <joevandyk> i’m using the ruby zmq gem. i’m using the req/res sockets. When i try to send a message after a client has disconnected and i’ve specified no blocking, the sender just blocks forever.
[Saturday 04 September 2010] [02:13:27] <joevandyk> I think I see the problem. When the service disconnects, the client still tries to send a message, and is waiting for a response.
[Saturday 04 September 2010] [02:13:34] <joevandyk> is there a way around that?
[Saturday 04 September 2010] [02:14:02] <joevandyk> i wonder if this is a bug in the ruby library
[Saturday 04 September 2010] [02:14:37] <lestrrat> sustrik: I sent a message to the mailing list, but I messed my From: field :/
[Saturday 04 September 2010] [02:15:11] <lestrrat> sustrik: anyway, I all I wanted to say was that the eintr branch worked :)
[Saturday 04 September 2010] [02:40:24] Quit bgranger has left this server (Quit: bgranger).
[Saturday 04 September 2010] [02:43:59] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [03:25:08] Quit keffo has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [03:39:46] Quit guido_g has left this server (Quit: Boom...).
[Saturday 04 September 2010] [03:51:24] <sustrik> lestrrat: great, thanks!
[Saturday 04 September 2010] [03:51:44] <sustrik> joevandyk: it’s the way 0mq works
[Saturday 04 September 2010] [03:52:26] <joevandyk> seems odd that if the service disconnects that the client is hung forever
[Saturday 04 September 2010] [03:52:55] <sustrik> it’ll continue working once service becaomes available
[Saturday 04 September 2010] [03:53:34] <sustrik> if you want to explicitly exit in case your task haven’t been completed in a predefined time, use timeouts
[Saturday 04 September 2010] [03:59:04] <joevandyk> sustrik: hm, i don’t see how it can become unblocked once it’s waiting for a recv
[Saturday 04 September 2010] [03:59:19] <sustrik> use zmq_poll
[Saturday 04 September 2010] [03:59:24] <sustrik> there’s a timout parameter there
[Saturday 04 September 2010] [03:59:41] <joevandyk> sustrik: what i’m not understanding is why the client sends a message when there’s no servers to receive it
[Saturday 04 September 2010] [04:00:13] <sustrik> what does “there’s no server” mean?
[Saturday 04 September 2010] [04:00:43] <sustrik> there’s no such thing in networking
[Saturday 04 September 2010] [04:01:10] <sustrik> what you have otoh is “the transfer haven’t been acknowledged in time”
[Saturday 04 September 2010] [04:01:29] <sustrik> which is the timeout
[Saturday 04 September 2010] [04:02:47] <joevandyk> sorry – when the REP socket disconnects. The REQ client had been happily sending data to the REP. But then the REP disconnected. The REQ tried to send data again to the REP and the send suceeded, but there was nothing to send it to.
[Saturday 04 September 2010] [04:19:08] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 04 September 2010] [04:37:29] Quit joevandyk has left this server (Quit: joevandyk).
[Saturday 04 September 2010] [06:09:07] Quit keffo has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [06:19:40] Quit keffo has left this server (Ping timeout: 258 seconds).
[Saturday 04 September 2010] [06:20:50] Join stephank has joined this channel (
~shteef@84.25.113.14).
[Saturday 04 September 2010] [06:22:11] Nick xla is now known as xla_soundcloud.
[Saturday 04 September 2010] [07:29:29] Nick xla_soundcloud is now known as xla_SC.
[Saturday 04 September 2010] [07:40:58] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [08:14:19] <Tasser> cremes, got me the big picture for zmqmachine?
[Saturday 04 September 2010] [08:24:33] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 04 September 2010] [08:33:10] <mato> re
[Saturday 04 September 2010] [08:33:44] <sustrik> hi
[Saturday 04 September 2010] [08:34:11] <mato> sustrik: list of patches to cherry-pick along with instructions coming up shortly
[Saturday 04 September 2010] [08:34:38] <sustrik> thx
[Saturday 04 September 2010] [08:34:40] <mato> sustrik: just need to do some housekeeping (also delete and recreate FD_SETSIZE issue in tracker since I found that bug, not pieter)
[Saturday 04 September 2010] [08:34:54] <sustrik> ok
[Saturday 04 September 2010] [08:42:42] <mato> ok, i can’t delete it, have commented with the current situation and removed the “maint” label
[Saturday 04 September 2010] [08:44:09] <sustrik> fine
[Saturday 04 September 2010] [08:49:44] <mato> sustrik: mumble... pieter also deleted the “empty” zmq_forwarder/streamer/queue documentation
[Saturday 04 September 2010] [08:49:57] <mato> sustrik: this has packaging implications
[Saturday 04 September 2010] [08:50:05] <mato> sustrik: how would you like me to resolve this?
[Saturday 04 September 2010] [08:50:10] <sustrik> get it back
[Saturday 04 September 2010] [08:50:31] <mato> ok, what about the zmq_device documentation he added?
[Saturday 04 September 2010] [08:50:46] <mato> are devices still experimental in 2.0.x?
[Saturday 04 September 2010] [08:51:34] <mato> or would you prefer to revert his commit entirely?
[Saturday 04 September 2010] [08:51:45] <mato> the problem is it has other stuff mixed up in it
[Saturday 04 September 2010] [08:51:59] <mato> like changes to asciidoc.conf which are half-broken
[Saturday 04 September 2010] [08:52:10] <sustrik> is there any problem with device documentation
[Saturday 04 September 2010] [08:52:29] <mato> i don’t know, but you were keeping it as undocumented
[Saturday 04 September 2010] [08:52:40] <mato> there is a problem with the other parts of the commit
[Saturday 04 September 2010] [08:53:22] <mato> like the asciidoc footer only being present for manpages, not for html
[Saturday 04 September 2010] [08:53:28] <sustrik> what’s easier for you?
[Saturday 04 September 2010] [08:54:22] <mato> less work right now would be just to revert the whole commit
[Saturday 04 September 2010] [08:54:49] <mato> and pieter can resubmit his changes
[Saturday 04 September 2010] [08:55:14] <mato> hmm, maybe
[Saturday 04 September 2010] [08:55:45] <sustrik> it’s up to you
[Saturday 04 September 2010] [09:15:30] Quit xla_SC has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [09:25:42] Quit rgl has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [10:03:23] <CIA-20> zeromq2: 03Martin Lucina 07master * r2673a84 10/ (34 files): (log message trimmed)
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: doc: Update zmq_socket(3) for 2.0.8 API changes
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Added man page for the zmq_device method”
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Added clean target that deletes generated man pages”
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Various changes to documentation project:”
[Saturday 04 September 2010] [10:03:34] <CIA-20> zeromq2: 03Martin Lucina 07maint * ree3444f 10/ doc/zmq_socket.txt : doc: Update zmq_socket(3) for 2.0.8 API changes (+8 more commits...) -
http://bit.ly/aRdJ8z[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: 03Martin Lucina 07master * rf850190 10/ include/zmq.h : zmq.h: Fix typo and use of C99 comment -
http://bit.ly/95mVlM[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: 03Martin Lucina 07master * r51a84c1 10/ (src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: Ensure that 0MQ does not attempt to call select() on more than FD_SETSIZE
[Saturday 04 September 2010] [10:25:09] <CIA-20> zeromq2: 03Martin Lucina 07master * rca17612 10/ (include/zmq.h src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:10] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [10:25:10] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [10:25:11] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:11] <CIA-20> zeromq2: zmq.h: Fix typo and use of C99 comment
[Saturday 04 September 2010] [10:25:12] <CIA-20> zeromq2: Conflicts:
[Saturday 04 September 2010] [10:25:15] <CIA-20> zeromq2: 03Martin Lucina 07maint * rf850190 10/ include/zmq.h : zmq.h: Fix typo and use of C99 comment -
http://bit.ly/95mVlM[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: 03Martin Lucina 07maint * r51a84c1 10/ (src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: Ensure that 0MQ does not attempt to call select() on more than FD_SETSIZE
[Saturday 04 September 2010] [10:43:08] Quit plq has left this server (Remote host closed the connection).
[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Ivo Danihelka 07maint * rae567be 10/ (AUTHORS src/zmq.cpp): improved null checking in zmq_term -
http://bit.ly/anJS6s[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Jon Dyte 07maint * rc2f3b3b 10/ (src/forwarder.cpp src/queue.cpp src/streamer.cpp): forwarder and streamer devices handle multi-part messages correctly -
http://bit.ly/aRxGoy[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Dhammika Pathirana 07maint * r1022789 10/ src/zmq_decoder.cpp : assert on malformed messages -
http://bit.ly/907fWH[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rdb7fe85 10/ include/zmq.h : Broken device numbering reverted -
http://bit.ly/cncpxn[Saturday 04 September 2010] [11:19:45] Quit poswald has left this server (Quit: poswald).
[Saturday 04 September 2010] [11:20:13] <CIA-20> zeromq2: 03Ivo Danihelka 07master * rae567be 10/ (AUTHORS src/zmq.cpp): improved null checking in zmq_term -
http://bit.ly/anJS6s[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Jon Dyte 07master * rc2f3b3b 10/ (src/forwarder.cpp src/queue.cpp src/streamer.cpp): forwarder and streamer devices handle multi-part messages correctly -
http://bit.ly/aRxGoy[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Dhammika Pathirana 07master * r1022789 10/ src/zmq_decoder.cpp : assert on malformed messages -
http://bit.ly/907fWH[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Martin Sustrik 07master * rdb7fe85 10/ include/zmq.h : Broken device numbering reverted -
http://bit.ly/cncpxn[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: 03Martin Sustrik 07master * r76f2e5d 10/ include/zmq.h : (log message trimmed)
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: Broken device numbering reverted
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: assert on malformed messages
[Saturday 04 September 2010] [11:20:25] <CIA-20> zeromq2: forwarder and streamer devices handle multi-part messages correctly
[Saturday 04 September 2010] [11:20:26] <CIA-20> zeromq2: improved null checking in zmq_term
[Saturday 04 September 2010] [11:22:45] Quit xla_SC has left this server (Ping timeout: 260 seconds).
[Saturday 04 September 2010] [11:34:39] <AndrewBC> and could use a fresh pair of eyes.
[Saturday 04 September 2010] [11:36:41] <AndrewBC> I also don’t see anything pertaining to this in the buglist
[Saturday 04 September 2010] [11:46:55] <AndrewBC> That gibberish output is looking more and more sinister, you know
[Saturday 04 September 2010] [11:47:00] <AndrewBC> I wonder what that’s about
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: 03Martin Lucina 07master * ra6d3629 10/ (Makefile.am configure.in ChangeLog): (log message trimmed)
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: build: Generate ChangeLog in ‘make dist’, ZIP automatically
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: Change ‘make dist’ to generate the Git ChangeLog file, that way it doesn’t
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: have to be manually updated nor kept in Git which causes unnecessary work.
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: Also change ‘make dist’ to invoke ‘dist-zip’ automatically to generate a
[Saturday 04 September 2010] [11:50:22] <CIA-20> zeromq2: ZIP as well as a .tar.gz.
[Saturday 04 September 2010] [11:50:23] <CIA-20> zeromq2: 03Martin Lucina 07master * r32fd916 10/ doc/asciidoc.conf :
[Saturday 04 September 2010] [11:50:24] <CIA-20> zeromq2: doc: Add 0MQ version to XHTML11 backend footer
[Saturday 04 September 2010] [11:50:24] <CIA-20> zeromq2: Thanks to Matt Weinstein for the suggestion. -
http://bit.ly/d1r24H[Saturday 04 September 2010] [11:50:25] <CIA-20> zeromq2: 03Martin Lucina 07master * r1e84519 10/ .gitignore : Update .gitignore -
http://bit.ly/bH7Jhp[Saturday 04 September 2010] [11:50:25] <CIA-20> zeromq2: 03Martin Lucina 07master * rd4c8de5 10/ (5 files in 2 dirs):
[Saturday 04 September 2010] [11:50:26] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [11:50:26] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [11:50:27] <CIA-20> zeromq2: Update .gitignore
[Saturday 04 September 2010] [11:50:27] <CIA-20> zeromq2: doc: Add 0MQ version to XHTML11 backend footer
[Saturday 04 September 2010] [11:54:08] <AndrewBC> Okay, I tracked it down to the zmq_recv() in my client. Perhaps there’s something special in the flags I need to be aware of since it’s on the other side of an XREP
[Saturday 04 September 2010] [11:54:30] Quit gebi has left this server (Read error: Operation timed out).
[Saturday 04 September 2010] [11:54:48] <AndrewBC> So I’m reading the API docs now
[Saturday 04 September 2010] [12:01:34] <AndrewBC> beginning to suspect that I should be checking for multipart messages
[Saturday 04 September 2010] [12:06:26] <sccolbert> Perhaps a simple problem, but I appear to have a mem leak on the SUB side of a PUB/SUB pattern
[Saturday 04 September 2010] [12:06:50] <sccolbert> when I connect the SUB to the PUB, any messages not matching the filter cause wasted memory
[Saturday 04 September 2010] [12:07:30] <sccolbert> a simple test send say, 1 million short messages that dont match any filter on the SUB, will cause the memory usage of the SUB to explode
[Saturday 04 September 2010] [12:07:55] <sccolbert> has anyone else experienced this?
[Saturday 04 September 2010] [12:08:47] <AndrewBC> I assume that since the filtering is done on the SUB side, the messages are indeed being transmitted to the socket on that end
[Saturday 04 September 2010] [12:09:50] <sccolbert> this is true, but their memory is never freed
[Saturday 04 September 2010] [12:10:13] <AndrewBC> Are you using a binding, or what?
[Saturday 04 September 2010] [12:10:22] <sccolbert> PyZMQ
[Saturday 04 September 2010] [12:10:28] <sccolbert> on ubuntu 10.04
[Saturday 04 September 2010] [12:11:00] <sccolbert> so if I just connect the SUB and dont set any filter, the SUB process quickly locks up the system
[Saturday 04 September 2010] [12:11:30] <sccolbert> which I cant see as being related to PyZMQ
[Saturday 04 September 2010] [12:12:15] <AndrewBC> I’m not sure, since I’m not a dev of zmq. It’s supposed to handle the memory just fine, if wishes were horses...
[Saturday 04 September 2010] [12:12:45] <AndrewBC> It could be either, though. All it takes is a circular reference for python to not garbage collect objects
[Saturday 04 September 2010] [12:13:11] <AndrewBC> circular, cyclic, bah
[Saturday 04 September 2010] [12:14:00] <sccolbert> this is true, though I would think that since I havent actually called recv() on the sub (since its filtering all messages) that python isnt even aware a message arrived
[Saturday 04 September 2010] [12:20:40] Quit xla_SC has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [12:21:46] <AndrewBC> Wish I could help you. I’m banging my head up against a bizzare failed assertion myself
[Saturday 04 September 2010] [12:34:40] Quit xla_SC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [12:38:37] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rebf3089 10/ NEWS : NEWS updated for 2.0.9 -
http://bit.ly/doVZJ4[Saturday 04 September 2010] [12:45:32] Quit AndrewBC has left this server (Ping timeout: 272 seconds).
[Saturday 04 September 2010] [12:56:38] <pieter_hintjens> sccolbert: what version of 0MQ are you using?
[Saturday 04 September 2010] [12:58:09] <sccolbert> pieter_hintjens: 2.0.7
[Saturday 04 September 2010] [12:58:50] <pieter_hintjens> when you stop the publisher does the memory consumption of the subscriber go down again?
[Saturday 04 September 2010] [12:59:03] <sccolbert> hmm havent tried that
[Saturday 04 September 2010] [12:59:05] <sccolbert> gimme a sec
[Saturday 04 September 2010] [13:01:36] <sccolbert> no
[Saturday 04 September 2010] [13:01:50] <sccolbert> killing the publisher does not free the memory in the SUB
[Saturday 04 September 2010] [13:04:10] <pieter_hintjens> hmm... so the next steps are to test with the latest 2.0.9 release
[Saturday 04 September 2010] [13:04:19] <pieter_hintjens> and then make a minimal test case that reproduces this, and log an issue
[Saturday 04 September 2010] [13:04:30] <sccolbert> k, lemme grab the new one along with the new PyZMQ
[Saturday 04 September 2010] [13:04:36] <sccolbert> will only take me a couple minutes
[Saturday 04 September 2010] [13:04:52] <sccolbert> a different question:
[Saturday 04 September 2010] [13:04:54] <pieter_hintjens> if you can make a parallel test case in C or C++ that will help identify whether it’s in PyZMQ or 0MQ
[Saturday 04 September 2010] [13:05:04] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [13:05:04] <sccolbert> sure
[Saturday 04 September 2010] [13:05:14] <pieter_hintjens> shoot...
[Saturday 04 September 2010] [13:05:45] <sccolbert> in a pub/sub pattern, if i send a multipart message, and the first part doesnt match the sub filter, does the sub still recieve the rest of the message?
[Saturday 04 September 2010] [13:06:00] <sccolbert> and then just dump it
[Saturday 04 September 2010] [13:06:06] <pieter_hintjens> i actually added this to the guide today but did not publish it yet
[Saturday 04 September 2010] [13:06:14] <pieter_hintjens> a multipart message is one message
[Saturday 04 September 2010] [13:06:23] <pieter_hintjens> think of the first part as the envelope
[Saturday 04 September 2010] [13:06:32] <pieter_hintjens> it’s very convenient to put the pubsub key there
[Saturday 04 September 2010] [13:06:47] <pieter_hintjens> if the key does not match any subscription the entire message is discarded / ignored
[Saturday 04 September 2010] [13:07:11] <sccolbert> is there any performance improvement to be had with that, over just joining my key to the beggininng of my message?
[Saturday 04 September 2010] [13:07:29] <pieter_hintjens> using a pubsub envelope like this means you can do zerocopy on the data part
[Saturday 04 September 2010] [13:07:41] <sccolbert> ah, great
[Saturday 04 September 2010] [13:07:50] <pieter_hintjens> there is no significant performance cost and you can save on formatting the full message
[Saturday 04 September 2010] [13:08:23] <pieter_hintjens> plus you are guaranteed the key remains separate from the data
[Saturday 04 September 2010] [13:08:29] <pieter_hintjens> it’s a Good Thing
[Saturday 04 September 2010] [13:08:40] <sccolbert> yep
[Saturday 04 September 2010] [13:09:01] <sccolbert> also, in a general pub/sub pattern do you recommend epgm over tcp?
[Saturday 04 September 2010] [13:09:13] <CIA-20> zeromq2: 03Martin Sustrik 07maint * r01c463c 10/ (builds/msvc/platform.hpp configure.in): Version number incremented to 2.0.10 -
http://bit.ly/9vL8xS[Saturday 04 September 2010] [13:09:16] <sccolbert> I wasnt able to find much literature on the pro’s vs cons of each
[Saturday 04 September 2010] [13:10:11] <pieter_hintjens> you’d only use pgm/epgm if you have a performance issue with tcp
[Saturday 04 September 2010] [13:10:23] <pieter_hintjens> there is ongoing work to make tcp pubsub much more efficient for large fanouts
[Saturday 04 September 2010] [13:10:37] <pieter_hintjens> in general use tcp unless you know why you need multicast
[Saturday 04 September 2010] [13:10:38] <sccolbert> cool
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rebf3089 10/ NEWS : NEWS updated for 2.0.9 -
http://bit.ly/doVZJ4[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r01c463c 10/ (builds/msvc/platform.hpp configure.in): Version number incremented to 2.0.10 -
http://bit.ly/9vL8xS[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rb4740c1 10/ NEWS : (log message trimmed)
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [13:10:48] <CIA-20> zeromq2: Version number incremented to 2.0.10
[Saturday 04 September 2010] [13:10:49] <CIA-20> zeromq2: NEWS updated for 2.0.9
[Saturday 04 September 2010] [13:10:49] <CIA-20> zeromq2: Conflicts:
[Saturday 04 September 2010] [13:10:50] <CIA-20> zeromq2: builds/msvc/platform.hpp
[Saturday 04 September 2010] [13:11:24] <Steve-o> usually tcp up to 4-5 peers is better than multicast for almost all scenarios
[Saturday 04 September 2010] [13:11:35] <sccolbert> good to know
[Saturday 04 September 2010] [13:11:40] <sccolbert> thanks
[Saturday 04 September 2010] [13:13:01] <Steve-o> usually NICs give you some form of TCP acceleration compared to none for UDP multicast
[Saturday 04 September 2010] [13:16:07] <sustrik> uh, 2.0.9 is out
[Saturday 04 September 2010] [13:16:38] * sustrik has enough, goes away to play moonlight sonata
[Saturday 04 September 2010] [13:47:53] <sccolbert> pieter_hintjens: tested new version in C and Python
[Saturday 04 September 2010] [13:47:57] <sccolbert> same issues
[Saturday 04 September 2010] [13:48:37] <sccolbert> where should I log the issue?
[Saturday 04 September 2010] [13:50:29] <sccolbert> ah, ok, it only happens in C when the sub hasnt set any filter
[Saturday 04 September 2010] [13:50:44] <sccolbert> setting a filter that doesnt match anything from the PUB solves the issue
[Saturday 04 September 2010] [13:50:50] <sccolbert> in C at least
[Saturday 04 September 2010] [13:51:23] <sccolbert> i will file this with Brian
[Saturday 04 September 2010] [14:18:12] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 04 September 2010] [14:25:46] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [14:41:13] Nick
AndrewBC_ is now known as AndrewBC.
[Saturday 04 September 2010] [15:01:00] <twcc> Hey all, I’m trying to figure out if zeromq ist the right tool for me. I want to have one “Server” and multiple client, which pushes messages to the server. I can’t find a pattern which fits really good - Request-reply would fit best, but as I understood I have to send a reply. - Is this case so trivial, that I should fall back to standard socket programming?
[Saturday 04 September 2010] [15:11:41] Quit xla_SC has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [15:37:54] <Steve-o> twcc: sounds like PUB/SUB
[Saturday 04 September 2010] [15:43:36] <twcc> Because of the unidirectional manner this would fit, but AFAIK the Publisher starts the socket and the subscriber has to connect?! - This could run into problems with a changing number of publisher, or can I connect them to one socket, and zmq handles this?
[Saturday 04 September 2010] [15:53:17] <Steve-o> You can connect multiple to one zmq socket as detailed in zmq_socket(3)
[Saturday 04 September 2010] [15:54:40] <Steve-o> there is currently no bus topology socket pattern
[Saturday 04 September 2010] [15:55:13] <Steve-o> you can jump down to raw PGM for that
[Saturday 04 September 2010] [15:57:09] <twcc> maybe i’m stuck to much to socket programming ... - so I can bind multiple publisher to the same socket/port and have one subscriber which processes the messages? That would be great and solves all my problems :-)
[Saturday 04 September 2010] [16:01:35] <kooroo> hey folks, I’m having trouble compiling zeromq on rh5 x86_64
[Saturday 04 September 2010] [16:01:52] <kooroo> .libs/libzmq_la-txwi.o: relocation R_X86_64_PC32 against
`pgm_rs_create’ can not be used when making a shared object; recompile with -fPIC is what I’m getting.
[Saturday 04 September 2010] [16:02:00] <kooroo> has anyone come across this before?
[Saturday 04 September 2010] [16:02:11] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [16:03:06] <Steve-o> twcc: for each subscriber you call zmq_connect for each publisher on one zmq socket
[Saturday 04 September 2010] [16:04:49] <Steve-o> kooroo: it is a bug in RHEL GCC, you can disable PGM or there is a flag you can set to disable visibility on the PGM internal API
[Saturday 04 September 2010] [16:07:08] <kooroo> Steve-o, is this the -DPGM_GNUC_INTERNAL flag?
[Saturday 04 September 2010] [16:07:29] <Steve-o> kooroo: yes, it needs to be “-DPGM_GNUC_INTERNAL=” for broken GCC, default “-DPGM_GNUC_INTERNAL=G_GNUC_INTERNAL”
[Saturday 04 September 2010] [16:08:36] <Steve-o> I think someone found a backported GCC version that works too on the list before
[Saturday 04 September 2010] [16:10:45] <kooroo> Steve-o: do you know offhand if upgrading gcc works?
[Saturday 04 September 2010] [16:11:07] <Steve-o> 4.1.2 is the affected version
[Saturday 04 September 2010] [16:12:36] <kooroo> Steve-o: thanks, Imma upgrade gcc and try it again.
[Saturday 04 September 2010] [16:12:54] <twcc> Hm, maybe I explained it bad - I have a changing number of processes (publisher) which have to send messages to one sink (daemon/subscriber), which handles these messages. A topologie where the daemon/subscriber has to connect to the publisher isn’t feasible. - So the client/server application would fit best, except that the server has to response to a message.
[Saturday 04 September 2010] [16:13:50] <kooroo> twcc, could you have them all subscribe to the same multicast stream and only the subscriber react to messages?
[Saturday 04 September 2010] [16:14:15] <kooroo> not the most efficient way, I admit.
[Saturday 04 September 2010] [16:17:35] <twcc> hm, funny idea, didn’t thought of it
[Saturday 04 September 2010] [16:19:05] <twcc> multicast is only possible on network connections?, because all the processes are running on the same computer - so i would like to use ipc
[Saturday 04 September 2010] [16:24:02] <Steve-o> multicast requires a network for reliable delivery, it’s pretty useless for ipc.
[Saturday 04 September 2010] [16:30:35] <Steve-o> I think as IPC is currently implemented using local sockets you would have to add a shared memory transport for localhost broadcast
[Saturday 04 September 2010] [16:32:58] <twcc> I think the multicast idea would only be a workaround - I would like to have a non-blocking “Request-reply” without reply pattern, but I can’t figure out, how to realize it with zeromq
[Saturday 04 September 2010] [16:36:14] <twcc> Thank you for your help - I will try if the request-reply pattern satisfy the requirements and handle the non-blocking by myself
[Saturday 04 September 2010] [16:43:48] <Steve-o> post a message on the list too, it sounds like a good idea for v3, I think 29West have a broadcast IPC transport too
[Saturday 04 September 2010] [16:50:21] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [16:54:11] Part twcc has left this channel.
[Saturday 04 September 2010] [16:58:57] Quit xla_SC has left this server (Quit: leaving).
[Saturday 04 September 2010] [17:55:35] Quit sccolbert has left this server (Remote host closed the connection).
[Saturday 04 September 2010] [18:23:00] Quit Odizes has left this server (Quit: Saliendo).
[Saturday 04 September 2010] [18:37:19] <DasIch> (obviously what i really did was a bit more complex but that what it boils down to)
[Saturday 04 September 2010] [18:40:56] <lestrrat> sustrik: so the eintr thins is going to be in 2.1?
[Saturday 04 September 2010] [19:01:44] <lestrrat> sustrik: And the ZMQ_FD stuff to get at the file descriptor – is that going to be in 2.1 as well?
[Saturday 04 September 2010] [19:07:22] Quit bgranger has left this server (Quit: bgranger).
[Saturday 04 September 2010] [19:13:20] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Saturday 04 September 2010] [20:01:46] Quit Guest26371 has left this server (Quit: Page closed).
[Saturday 04 September 2010] [20:15:23] Quit DasIch has left this server (Ping timeout: 255 seconds).
[Saturday 04 September 2010] [21:13:32] Quit andrewvc has left this server (Quit: andrewvc).
[Saturday 04 September 2010] [21:45:06] Quit andrewvc has left this server (Quit: andrewvc).
[Saturday 04 September 2010] [21:49:55] Quit xla has left this server (Quit: leaving).
[Saturday 04 September 2010] [22:12:10] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [22:34:34] Quit plq has left this server (Quit: Leaving).
[Saturday 04 September 2010] [22:43:18] Quit jonrafkind has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [23:15:29] Quit sd88g93 has left this server (Quit: Ex-Chat).
[Saturday 04 September 2010] [23:21:28] Quit jonrafkind has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [01:40:12] <sustrik> lestrrat: both are going to be in 2.1
[Sunday 05 September 2010] [01:40:45] <sustrik> ZMQ_FD as already there
[Sunday 05 September 2010] [01:41:00] <sustrik> i’ll write an email about EINTR & 2.1 today
[Sunday 05 September 2010] [02:00:28] <sustrik> DasIch: That’s a bug in OpenPGM library
[Sunday 05 September 2010] [02:00:53] <sustrik> if you don’t need PGM, just compile 0MQ without it
[Sunday 05 September 2010] [02:01:36] <sustrik> it have been fixed in newer OpenPGM releases
[Sunday 05 September 2010] [02:01:44] <sustrik> but haven’t got into 0MQ as for now
[Sunday 05 September 2010] [02:01:59] <sustrik> anway, creating multiple contexts is pretty useless
[Sunday 05 September 2010] [02:02:08] <sustrik> why do you do that?
[Sunday 05 September 2010] [02:32:34] <lestrrat> sustrik: by “it’s already there” you mean the github repo, right?
[Sunday 05 September 2010] [02:34:06] <sustrik> yes
[Sunday 05 September 2010] [02:34:09] <sustrik> the master branch
[Sunday 05 September 2010] [02:34:33] <sustrik> EINTR on eintr branch, FD & EVENTS on master branch
[Sunday 05 September 2010] [02:35:21] <sustrik> if i have some spare time today, i’ll merge EINTR fix to master as well
[Sunday 05 September 2010] [02:57:14] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 05 September 2010] [03:29:53] Nick
AndrewBC_ is now known as AndrewBC.
[Sunday 05 September 2010] [03:57:15] Topic The channel topic is “Welcome!”.
[Sunday 05 September 2010] [03:57:21] Mode Channel modes: no messages from outside, topic protection
[Sunday 05 September 2010] [03:57:21] Created This channel was created on 2010-02-05 10:44.
[Sunday 05 September 2010] [03:57:29] Quit travlr has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [04:17:47] Quit kooroo has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [04:17:47] Nick kooroox2 is now known as kooroo.
[Sunday 05 September 2010] [04:39:40] <pieter_hintjens> sustrik: hi
[Sunday 05 September 2010] [05:50:29] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 05 September 2010] [05:54:31] <sustrik> pieter_hintjens: hi
[Sunday 05 September 2010] [05:57:29] <pieter_hintjens> hi martin... i was wondering about different ways to submit patches
[Sunday 05 September 2010] [05:57:47] <pieter_hintjens> people are using at least three different routes
[Sunday 05 September 2010] [05:57:59] <pieter_hintjens> patches to zeromq-dev, issues, and pull requests
[Sunday 05 September 2010] [05:58:48] <sustrik> i am using patches from the mailing list
[Sunday 05 September 2010] [05:58:56] <sustrik> are there any pending patches anywhere else?
[Sunday 05 September 2010] [05:59:05] <pieter_hintjens> i saw one pull request on the ML today
[Sunday 05 September 2010] [05:59:22] <sustrik> yup
[Sunday 05 September 2010] [05:59:35] <sustrik> that’s a packaging script thing, it’s up to mato to manage that one
[Sunday 05 September 2010] [05:59:47] * sustrik has no idea how packaging works
[Sunday 05 September 2010] [06:00:04] <pieter_hintjens> :-) it’s more of a consistency thing than anything else
[Sunday 05 September 2010] [06:00:27] <pieter_hintjens> ok, totally different thing, i have a trivial inproc example that refuses to work
[Sunday 05 September 2010] [06:00:39] <sustrik> can you paste is somewhere?
[Sunday 05 September 2010] [06:00:51] <pieter_hintjens> yes, i raised an issue (62) with an example and analysis
[Sunday 05 September 2010] [06:01:00] <sustrik> let me see
[Sunday 05 September 2010] [06:01:56] <pieter_hintjens> i’ve spent several hours trying different things, ended up doing a sleep... :-/
[Sunday 05 September 2010] [06:03:27] <sustrik> i would say it’s the inproc reconnect thing
[Sunday 05 September 2010] [06:03:40] <sustrik> inproc reconnect is still unimplemented
[Sunday 05 September 2010] [06:03:58] <sustrik> thus doing connect before bind does not work
[Sunday 05 September 2010] [06:04:34] <sustrik> so, if step1 connects before step2 binds
[Sunday 05 September 2010] [06:04:38] <sustrik> :|
[Sunday 05 September 2010] [06:04:39] <pieter_hintjens> ah, it’s not just for PAIR sockets
[Sunday 05 September 2010] [06:05:08] <sustrik> it’s inproc as well
[Sunday 05 September 2010] [06:05:18] <pieter_hintjens> with PUSH/PULL it works a little better, say 90% of the time
[Sunday 05 September 2010] [06:05:19] <sustrik> it’s not that visible as inproc connection cannot break
[Sunday 05 September 2010] [06:05:25] <pieter_hintjens> with PAIR it dies almost immediately
[Sunday 05 September 2010] [06:05:35] <sustrik> it’s a timing issue
[Sunday 05 September 2010] [06:05:44] <pieter_hintjens> ok, if this is a known restriction that’s fine, I can document it
[Sunday 05 September 2010] [06:06:19] <sustrik> Connecting a socket
[Sunday 05 September 2010] [06:06:19] <sustrik> When connecting a socket to a peer address using zmq_connect() with the inproc transport, the endpoint shall be interpreted as an arbitrary string identifying the name to connect to. The name
[Sunday 05 September 2010] [06:06:19] <sustrik> must have been previously created by assigning it to at least one socket within the same 0MQ context as the socket being connected.
[Sunday 05 September 2010] [06:06:19] <pieter_hintjens> there is no technical reason it could not work, right?
[Sunday 05 September 2010] [06:06:25] <sustrik> no
[Sunday 05 September 2010] [06:06:26] <pieter_hintjens> hmm, ok
[Sunday 05 September 2010] [06:06:32] <sustrik> just not implemented
[Sunday 05 September 2010] [06:06:39] <pieter_hintjens> excellent, that is all I need
[Sunday 05 September 2010] [06:06:44] <pieter_hintjens> i’ll close the issue
[Sunday 05 September 2010] [06:06:47] <pieter_hintjens> thx
[Sunday 05 September 2010] [06:06:50] <sustrik> np
[Sunday 05 September 2010] [06:08:10] <pieter_hintjens> “ZeroMQ is quickly becoming an even bigger hammer in the premature optimization planet of Newbo-Thumbia.”
[Sunday 05 September 2010] [06:08:24] <pieter_hintjens> vs. “Then they’d better find a new rite of passage, because ZeroMQ is turning into a category killer.”
[Sunday 05 September 2010] [06:08:36] <sustrik> :)
[Sunday 05 September 2010] [06:08:53] <pieter_hintjens> that is an excellent discussion...
[Sunday 05 September 2010] [06:09:00] <sustrik> yes, i liked it
[Sunday 05 September 2010] [06:09:20] <pieter_hintjens> i’m wondering if it’s worth resurrecting the old amqp file transfer semantics as an example over 0MQ
[Sunday 05 September 2010] [06:09:26] <pieter_hintjens> should be nice and easy to implement
[Sunday 05 September 2010] [06:09:41] <sustrik> if you have resources to do that
[Sunday 05 September 2010] [06:09:58] <sustrik> an example?
[Sunday 05 September 2010] [06:10:00] <pieter_hintjens> well, it’s a question of what examples are the most valuable
[Sunday 05 September 2010] [06:10:24] <pieter_hintjens> would you rather have a name resolution service or a file transfer service over 0MQ?
[Sunday 05 September 2010] [06:10:27] <sustrik> ah, in the guide?
[Sunday 05 September 2010] [06:10:29] <pieter_hintjens> yeah
[Sunday 05 September 2010] [06:10:43] <sustrik> afaiu the examples in the guide are to be simple
[Sunday 05 September 2010] [06:10:53] <sustrik> file transfer is rather complex
[Sunday 05 September 2010] [06:10:56] <pieter_hintjens> sustrik: with 0MQ everything is simple :-)
[Sunday 05 September 2010] [06:11:03] <sustrik> i mean chopping the files
[Sunday 05 September 2010] [06:11:09] <sustrik> reassemblig them
[Sunday 05 September 2010] [06:11:14] <sustrik> etc.
[Sunday 05 September 2010] [06:11:38] <pieter_hintjens> maybe
[Sunday 05 September 2010] [06:12:00] <pieter_hintjens> actually it’s not that complex, really
[Sunday 05 September 2010] [06:12:06] <sustrik> it’s up to you
[Sunday 05 September 2010] [06:12:17] <pieter_hintjens> the sophisticated part is the stage / delivery semantics but that’s a neat demo of 0MQ IMO
[Sunday 05 September 2010] [06:12:24] <pieter_hintjens> we’ll see...
[Sunday 05 September 2010] [06:12:33] <sustrik> optinally, it can be a tutorial
[Sunday 05 September 2010] [06:12:40] <pieter_hintjens> yes
[Sunday 05 September 2010] [06:12:52] <pieter_hintjens> it reminded me that someone said “80% of messaging is still file transfer”
[Sunday 05 September 2010] [06:13:33] <pieter_hintjens> and it kind of seems that by covering the extreme other side of messaging (from low-latency) we prove the generality of the model
[Sunday 05 September 2010] [06:14:01] <pieter_hintjens> also it fits into the notion of distributed device management via shuffling config files around
[Sunday 05 September 2010] [06:14:05] * pieter_hintjens will think about it
[Sunday 05 September 2010] [06:14:11] <sustrik> ok
[Sunday 05 September 2010] [06:14:39] <pieter_hintjens> we will need a simple tutorial on how to make and submit git patches
[Sunday 05 September 2010] [06:14:57] <sustrik> enumerating the steps won’t do?
[Sunday 05 September 2010] [06:15:07] <pieter_hintjens> that would be a simple tutorial, then
[Sunday 05 September 2010] [06:15:10] <pieter_hintjens> it would do, yes
[Sunday 05 September 2010] [06:15:35] <sustrik> gitt diff, send as a main, put [PATCH] in the subject, state it’s under MIT
[Sunday 05 September 2010] [06:15:43] <sustrik> that’s it, no?
[Sunday 05 September 2010] [06:15:59] <sustrik> git, mail*
[Sunday 05 September 2010] [06:16:12] <pieter_hintjens> roughly, but some of the details cause problems
[Sunday 05 September 2010] [06:16:32] <pieter_hintjens> “Honestly? I spent too much time being stuck. I ended up on some crappy tutorial that said to type “git-format-patch”, when of course it was “git format-patch”. I got stuck on that until I trial-and-error’d.”
[Sunday 05 September 2010] [06:16:58] <sustrik> so far we don’t require formatted patches
[Sunday 05 September 2010] [06:17:05] <sustrik> so simple diff would do
[Sunday 05 September 2010] [06:17:11] <sustrik> not even git diff is needed
[Sunday 05 September 2010] [06:17:22] <sustrik> but it would be nice if people sent formatted patches
[Sunday 05 September 2010] [06:17:22] <pieter_hintjens> mato was saying something about signed-off-by
[Sunday 05 September 2010] [06:17:31] <sustrik> yup
[Sunday 05 September 2010] [06:17:33] <pieter_hintjens> indeed
[Sunday 05 September 2010] [06:17:56] <pieter_hintjens> for later...
[Sunday 05 September 2010] [06:18:14] <sustrik> would be nice to document it though
[Sunday 05 September 2010] [06:18:35] <pieter_hintjens> yes
[Sunday 05 September 2010] [06:18:46] <pieter_hintjens> did you update the contract page or shall I do that
[Sunday 05 September 2010] [06:18:53] <pieter_hintjens> i thought the way you expressed it in the email was very clear
[Sunday 05 September 2010] [06:19:07] <pieter_hintjens> but it needs to be written down, or it’ll get lost
[Sunday 05 September 2010] [06:20:16] <sustrik> ok, i’ll do that
[Sunday 05 September 2010] [06:20:44] <pieter_hintjens> what we found, in FFII work, was that it was usually better to edit the page first, then post to the list explaining the change
[Sunday 05 September 2010] [06:21:03] <pieter_hintjens> this is kind of a standard pattern to avoid losing stuff in the email list
[Sunday 05 September 2010] [06:25:33] <sustrik> ok
[Sunday 05 September 2010] [06:25:43] <sustrik> btw, the inproc example
[Sunday 05 September 2010] [06:26:04] <sustrik> you can fix it by launch step1’s thread from step2
[Sunday 05 September 2010] [06:26:13] <sustrik> that should synchronise it
[Sunday 05 September 2010] [06:27:39] <pieter_hintjens> hmm, nice solution...
[Sunday 05 September 2010] [06:28:50] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [06:31:39] <pieter_hintjens> ack, that works
[Sunday 05 September 2010] [06:31:45] <sustrik> great
[Sunday 05 September 2010] [06:35:44] <guido_g> hi
[Sunday 05 September 2010] [06:35:58] <guido_g> just saw the patch thing
[Sunday 05 September 2010] [06:36:25] <guido_g> I used git format-patch
[Sunday 05 September 2010] [06:36:44] <guido_g> but i had my changes in a local branch
[Sunday 05 September 2010] [06:37:52] <sustrik> guido_g: hi
[Sunday 05 September 2010] [06:37:59] <sustrik> i am not an expert on git
[Sunday 05 September 2010] [06:38:12] <sustrik> do you need a local branch to be able to format a patch?
[Sunday 05 September 2010] [06:38:47] <guido_g> i’m no expert neither, but will have a look
[Sunday 05 September 2010] [06:39:20] <sustrik> looks like there’s a “single-commit” mode
[Sunday 05 September 2010] [06:42:52] <guido_g> git format-patch -k –stdout R1..R2 <- from the examples of git help format-patch
[Sunday 05 September 2010] [06:43:08] <guido_g> seems that it works for commits and ranges of commits
[Sunday 05 September 2010] [06:43:32] <sustrik> nice
[Sunday 05 September 2010] [06:43:38] <guido_g> git format-patch -3 <- sweet
[Sunday 05 September 2010] [06:43:54] <guido_g> last three commits as patches
[Sunday 05 September 2010] [06:44:22] <sustrik> bingo
[Sunday 05 September 2010] [06:44:28] <sustrik> that’s what most people need
[Sunday 05 September 2010] [06:44:36] <guido_g> uh
[Sunday 05 September 2010] [06:45:06] * guido_g thinks changes should be done in branches
[Sunday 05 September 2010] [06:46:11] <sustrik> they should, but you don’t expect newbies submitting patches to deal with branching
[Sunday 05 September 2010] [06:46:29] <sustrik> installing git and learning how to commit is hard enough
[Sunday 05 September 2010] [06:46:54] <sustrik> that’s why i am happy with applying simple git diffs for now
[Sunday 05 September 2010] [06:47:20] <sustrik> or even dumb old diffs
[Sunday 05 September 2010] [06:47:55] <guido_g> ok
[Sunday 05 September 2010] [06:48:07] <sustrik> step-by-step would be nice though
[Sunday 05 September 2010] [06:48:08] <guido_g> so I spend hours for nothing...
[Sunday 05 September 2010] [06:48:10] <guido_g> :)
[Sunday 05 September 2010] [06:48:18] <sustrik> i mean, that’s the goal
[Sunday 05 September 2010] [06:48:25] <sustrik> we have to educate people
[Sunday 05 September 2010] [06:48:29] <guido_g> right
[Sunday 05 September 2010] [06:48:47] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [06:48:48] <sustrik> do we have a patch process described anywhere?
[Sunday 05 September 2010] [06:48:57] <sustrik> hm
[Sunday 05 September 2010] [06:49:05] * guido_g runs for cover
[Sunday 05 September 2010] [06:49:59] <pieter_hintjens> gudio_g: i think branches are orthogonal to patches
[Sunday 05 September 2010] [06:50:11] <pieter_hintjens> in the simplest case, you clone a repo, make changes, do a diff or git diff, and get a patch
[Sunday 05 September 2010] [06:50:34] <pieter_hintjens> well, that’s the medium hard case
[Sunday 05 September 2010] [06:50:53] <pieter_hintjens> in the simplest case you take the source code package, make changes, do a diff and get a non-git patch
[Sunday 05 September 2010] [06:51:13] <pieter_hintjens> but that only applies to projects that make source code packages
[Sunday 05 September 2010] [06:51:20] <sustrik> ack, and the hard case requires a topc branch
[Sunday 05 September 2010] [06:51:26] <sustrik> topic*
[Sunday 05 September 2010] [06:52:27] <pieter_hintjens> yes, exactly
[Sunday 05 September 2010] [06:53:07] <pieter_hintjens> for projects like zguide, there is no source package
[Sunday 05 September 2010] [06:53:24] <pieter_hintjens> we have to educate people, as you say
[Sunday 05 September 2010] [06:54:05] <sustrik> we are at the beginning of the road, it’ll take some time to make this work
[Sunday 05 September 2010] [06:54:13] <sustrik> but it’s worth of doing
[Sunday 05 September 2010] [06:55:38] <sustrik> guido_g: can you write down your experience with format-patch?
[Sunday 05 September 2010] [06:56:35] <sustrik> it could be used as a basis for the patch process tutorial
[Sunday 05 September 2010] [06:57:54] <guido_g> hmmm
[Sunday 05 September 2010] [06:58:11] <guido_g> i did it by book, three steps and done
[Sunday 05 September 2010] [06:58:22] <guido_g> but if you really want it...
[Sunday 05 September 2010] [06:58:37] <sustrik> yes, please
[Sunday 05 September 2010] [06:58:43] <sustrik> you’ll spare me an hour :)
[Sunday 05 September 2010] [06:58:46] <pieter_hintjens> guido_g: yes please, we’ll have to document the whole thing
[Sunday 05 September 2010] [06:58:50] <guido_g> sigh
[Sunday 05 September 2010] [06:58:52] <pieter_hintjens> how to submit patch from source package
[Sunday 05 September 2010] [06:59:09] <pieter_hintjens> how to submit patch from git repository clone
[Sunday 05 September 2010] [06:59:12] <guido_g> ok, then i’ll write down what I did
[Sunday 05 September 2010] [06:59:17] <sustrik> thanks
[Sunday 05 September 2010] [06:59:18] <pieter_hintjens> how to make topic branches and then submit them as patches
[Sunday 05 September 2010] [06:59:37] <pieter_hintjens> can you create a page on the wiki, minimal, and we can take it from there?
[Sunday 05 September 2010] [06:59:52] <pieter_hintjens> you should be able to create pages in the docs: category
[Sunday 05 September 2010] [07:00:06] <pieter_hintjens> “we” = everyone who cares about this
[Sunday 05 September 2010] [07:01:02] <guido_g> this cross-domain cookie thing sucks
[Sunday 05 September 2010] [07:02:38] <sustrik> ah, yes
[Sunday 05 September 2010] [07:02:53] <sustrik> pieter, any idea what’s the problem?
[Sunday 05 September 2010] [07:02:56] <guido_g> ok, where’s the how-to for the wiki?
[Sunday 05 September 2010] [07:03:09] <guido_g> sustrik: it’s a wikidot thing
[Sunday 05 September 2010] [07:03:12] <sustrik> guido_g: just post the command lines here
[Sunday 05 September 2010] [07:03:18] <guido_g> ok
[Sunday 05 September 2010] [07:11:33] <guido_g> that’s the whole magic
[Sunday 05 September 2010] [07:12:26] <guido_g> “git format-patch -1” is really nice
[Sunday 05 September 2010] [07:13:22] <sustrik> thanks!
[Sunday 05 September 2010] [07:16:40] <jond> hi
[Sunday 05 September 2010] [07:16:47] <sustrik> jon: hi
[Sunday 05 September 2010] [07:16:59] <jond> good to see a policy round the patches
[Sunday 05 September 2010] [07:17:11] <sustrik> ack
[Sunday 05 September 2010] [07:17:12] <sustrik> will you be able to make it for the meetup on tuesday?
[Sunday 05 September 2010] [07:17:35] <jond> I’ve had trouble squahsing commits in the past, which are meant to be easier for maintainer
[Sunday 05 September 2010] [07:17:44] <sustrik> i would like to have a talk with you about the subscription forwarding
[Sunday 05 September 2010] [07:18:07] <jond> tues: should be ok, though had to re-arrange a family thing
[Sunday 05 September 2010] [07:18:20] <sustrik> ah, sorry about that
[Sunday 05 September 2010] [07:19:16] <sustrik> discussing development via emails takes much more time than having a private talk
[Sunday 05 September 2010] [07:19:16] <jond> also; i can’t see irc in the dayjob, or twitter etc so if there’s any last minute change mailing list is best
[Sunday 05 September 2010] [07:19:41] <sustrik> jond: ok, will do
[Sunday 05 September 2010] [07:21:22] <jond> on the subject of git, the chacon book seems to recommend git pull requests and format patch. it’s all very powerful but seems easy to hang yrself
[Sunday 05 September 2010] [07:22:08] <sustrik> we’ll refine the process as we get better in using git
[Sunday 05 September 2010] [07:22:09] <guido_g> format-patch itself is quite nice, as shown above
[Sunday 05 September 2010] [07:22:35] <jond> am I meant to bring my local patch branches upto date with master/maint before running the diff/format-patch?
[Sunday 05 September 2010] [07:23:01] <sustrik> jond: that would be nice
[Sunday 05 September 2010] [07:23:31] <sustrik> otherwise it’s up to me to resolve the conflicts
[Sunday 05 September 2010] [07:23:40] <sustrik> which can in turn break your code
[Sunday 05 September 2010] [07:24:12] <sustrik> well, it depends
[Sunday 05 September 2010] [07:24:25] <sustrik> if you are submitting a single-line patch
[Sunday 05 September 2010] [07:24:34] <sustrik> and there have been a lot of change on master
[Sunday 05 September 2010] [07:24:46] <sustrik> it’s probably easier for me to merge it then for you
[Sunday 05 September 2010] [07:24:50] <jond> I also think we should recommend squashing the commits, as it makes the patch simpler to read (if git rebase -i works that is)
[Sunday 05 September 2010] [07:25:25] <sustrik> squash = rebase to the actual head?
[Sunday 05 September 2010] [07:25:55] <guido_g> already going way to far
[Sunday 05 September 2010] [07:26:09] <jond> i am trying to keep in line with 2.0.9 and maint at the moment. what mato seems to have put in place is similar to waht we do with clearcase in the day job.
[Sunday 05 September 2010] [07:26:41] <guido_g> pull -> [branch or not] -> changes -> commit [may be repeated] -> format-patch -> mailing list
[Sunday 05 September 2010] [07:27:59] <jond> well in the past, I used the rebase to make all the individual commits I did on my branch appear as 1 for delivery. It’s really annoying to have seperate commits for typo’s etc
[Sunday 05 September 2010] [07:28:41] <guido_g> but it’s great if you have different changes made in a row
[Sunday 05 September 2010] [07:28:52] <guido_g> so the maintainer can the ones he likes
[Sunday 05 September 2010] [07:29:25] <guido_g> anyhow, i think i’ll stick with this easy way
[Sunday 05 September 2010] [07:29:55] <jond> i’d use a seperate branch for each change.
[Sunday 05 September 2010] [07:30:33] <guido_g> if you do large changes, that might be ok
[Sunday 05 September 2010] [07:30:53] <guido_g> for smaller ones it’s sounds like a nightmare
[Sunday 05 September 2010] [07:32:32] <sustrik> jond: i wouldn’t bother with maint branch for the development work
[Sunday 05 September 2010] [07:32:54] <sustrik> it’s meant rather for small fixes
[Sunday 05 September 2010] [07:34:04] <sustrik> guido_g: yes, the smaller the patch the simpler the process should be
[Sunday 05 September 2010] [07:34:26] <sustrik> that’s why i’m saying i would accept even plain old diffs
[Sunday 05 September 2010] [07:34:45] <guido_g> great
[Sunday 05 September 2010] [07:35:02] <guido_g> lowers the entry barrier
[Sunday 05 September 2010] [07:35:11] <sustrik> exactly
[Sunday 05 September 2010] [07:35:27] <sustrik> however, if you ar woking on new functionality that’ll take 1/2 yesr to complete
[Sunday 05 September 2010] [07:35:39] <sustrik> you definitely want your own branch
[Sunday 05 September 2010] [07:35:44] <guido_g> and i’d demand payment :)
[Sunday 05 September 2010] [07:35:51] <sustrik> good point :)
[Sunday 05 September 2010] [07:37:28] <guido_g> so, now that i burned my eggs for brunch while writing git examples, i’m in the right mood for writing some more tests...
[Sunday 05 September 2010] [07:37:38] <jond> the many branches thing pretty much also depends on the turnaround of the patch getting to master etc
[Sunday 05 September 2010] [07:38:40] <jond> guido_g : i should send you the prefix_tree test ( was on the mailing list, i think we need tests now)
[Sunday 05 September 2010] [07:39:19] <jond> sustrik: hopefully get chance to discuss that pub filtering on tuesday
[Sunday 05 September 2010] [07:41:12] <sustrik> jond: sorry, i’ve seen it, but i was busy since :(
[Sunday 05 September 2010] [07:42:10] Quit xla has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [07:43:41] <jond> yes me too; 0MQ stuff very intermittent , as dayjob is not involving it all.
[Sunday 05 September 2010] [07:44:13] <guido_g> sigh at least, I’m not alone...
[Sunday 05 September 2010] [07:44:23] <jond> incidently a SUB socket can bind can’t it and then mutiple PUB’s connect
[Sunday 05 September 2010] [07:46:35] <sustrik> jond: yes
[Sunday 05 September 2010] [07:47:13] <sustrik> thus the subscription should be forwarded to _all_ connected publishers
[Sunday 05 September 2010] [07:47:51] <jond> ok, then maybe that’s one for the docs, someone asked that last night on here, they wanted fan-in to single server, I don’t think we cover that in the docs.
[Sunday 05 September 2010] [07:48:39] <sustrik> it kind of follows from the orthogonality of socket types and binds/connects
[Sunday 05 September 2010] [07:48:46] Quit keffo has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [07:48:53] <sustrik> but yes, it would be nice to mention it somewhere
[Sunday 05 September 2010] [07:49:02] <sustrik> maybe in the guide?
[Sunday 05 September 2010] [07:49:40] <jond> sustrik: sorry I meant the guide
[Sunday 05 September 2010] [07:50:09] <sustrik> pieter_hintjens: see above, suggestion for the guide
[Sunday 05 September 2010] [07:51:19] <sustrik> btw, the fact that binds/connects are independent of socket types is something that most people don’t get immediately
[Sunday 05 September 2010] [07:55:22] <jond> yes, i think that is the problem. there’s that big book enterprise messaging or whatever be nice to lift some scenarios from there and put in guide in 0MQ terms, which is pretty much what ph has been doing very well.
[Sunday 05 September 2010] [07:56:23] <sustrik> yes, it seems to work
[Sunday 05 September 2010] [07:57:08] <sustrik> btw, i’ve already seen some job ads requiring 0MQ knowledge so hopefully we’ll get into the phase where people are actually paid for working on 0MQ
[Sunday 05 September 2010] [07:57:11] <sustrik> we’ll see
[Sunday 05 September 2010] [07:58:55] <jond> that’s interesting news. I can see it being difficult to get 0MQ in some places with established code (understandly so in some cases, like reliable multicast)
[Sunday 05 September 2010] [08:01:48] <sustrik> well, i don’t expect nasdaq to replace their infrastrucutre by 0mq any time soon :)
[Sunday 05 September 2010] [08:02:11] <sustrik> but small companies are more flexible in adopting new technologies
[Sunday 05 September 2010] [08:05:11] <guido_g> hopefully
[Sunday 05 September 2010] [08:05:37] <guido_g> would be nice to have a mq related day job
[Sunday 05 September 2010] [08:09:08] <sustrik> my guess is that the early adopters are stock trading start-ups
[Sunday 05 September 2010] [08:09:46] <sustrik> that would mean that places like NY, London or Chicago are most interesting
[Sunday 05 September 2010] [08:09:55] <sustrik> i may be wrong though
[Sunday 05 September 2010] [08:10:35] <guido_g> hmmm... with mongrel2 there might be even ordinary web-work available
[Sunday 05 September 2010] [08:10:51] <sustrik> interesting point
[Sunday 05 September 2010] [08:11:11] <sustrik> i haven’t thought of that as i never done anything with web myself
[Sunday 05 September 2010] [08:11:29] <guido_g> i left it in early 2005 and never looked back :)
[Sunday 05 September 2010] [08:11:41] <sustrik> :)
[Sunday 05 September 2010] [08:11:44] <guido_g> never did fronted things anyway
[Sunday 05 September 2010] [08:12:13] <lestrrat> I doubt mongrel2 would really generate that much 0mq jobs... not at least from the web developer’s standpoint.
[Sunday 05 September 2010] [08:12:37] <lestrrat> zeromq is completely abstracted away in the web framework layer, so the web developers won’t even know ;)
[Sunday 05 September 2010] [08:12:57] <sustrik> right – unless you want to scale
[Sunday 05 September 2010] [08:13:06] <sustrik> what if you want N web servers
[Sunday 05 September 2010] [08:13:07] <lestrrat> of course, some of the more curious guys will do interesting web stuff with 0mq
[Sunday 05 September 2010] [08:13:14] <sustrik> and load-balanced services
[Sunday 05 September 2010] [08:13:22] <sustrik> persistence in the middle, etc.
[Sunday 05 September 2010] [08:13:40] <sustrik> that are google-like scenarios afaics
[Sunday 05 September 2010] [08:14:04] <guido_g> would also be nice to have it for mor ebay like things
[Sunday 05 September 2010] [08:14:08] <guido_g>
*more
[Sunday 05 September 2010] [08:14:14] <lestrrat> well, you’re talking about the smart guys, they’ll do anything. your average joe web devs won’t touch it with a 40ft pole
[Sunday 05 September 2010] [08:14:16] <lestrrat> ;)
[Sunday 05 September 2010] [08:14:36] <guido_g> lestrrat: you’re wrong
[Sunday 05 September 2010] [08:14:42] <guido_g> they do touch it
[Sunday 05 September 2010] [08:14:48] <guido_g> which is the problem...
[Sunday 05 September 2010] [08:15:05] <lestrrat> I knew where that was going ;)
[Sunday 05 September 2010] [08:15:17] <sustrik> :)
[Sunday 05 September 2010] [08:15:38] <guido_g> hehe
[Sunday 05 September 2010] [08:24:37] Quit keffo has left this server (Ping timeout: 260 seconds).
[Sunday 05 September 2010] [08:31:23] <guido_g> q: where do i find the declaration of int64_t?
[Sunday 05 September 2010] [08:33:24] <sustrik> guido_g: stdint.h
[Sunday 05 September 2010] [08:33:36] <guido_g> thx
[Sunday 05 September 2010] [09:06:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [09:15:09] <jond> sustrik: yes, for the large orgs change more likely thru acquisition/merger so small is best for more leading stuff
[Sunday 05 September 2010] [09:16:08] <jond> re : mongrel2 does anyone understand how the 0MQ integration works? It uses that c based co-routine library?
[Sunday 05 September 2010] [09:18:08] Quit xla has left this server (Read error: Operation timed out).
[Sunday 05 September 2010] [09:37:22] <sustrik> jond: i think so, but i haven’t checked
[Sunday 05 September 2010] [11:02:38] Quit
xla_ has left this server (Ping timeout: 245 seconds).
[Sunday 05 September 2010] [11:12:42] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [11:40:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [11:54:32] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [11:57:20] Quit xla has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [12:27:54] Part andrewvc has left this channel.
[Sunday 05 September 2010] [12:31:10] Quit AndrewBC has left this server (Ping timeout: 272 seconds).
[Sunday 05 September 2010] [12:37:26] Quit keffo has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [13:25:56] <cremes> Tasser: i’m not on irc very much during the weekends (Chicago, IL, US); if you have questions i recommend you send them to the mailing list
[Sunday 05 September 2010] [13:37:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [14:05:27] <pieter_hintjens> sustrik: re, was out with the family in the woods...
[Sunday 05 September 2010] [14:08:03] <pieter_hintjens> i will cover publishers connecting to subscriber when i find a sensible use case
[Sunday 05 September 2010] [14:08:28] <pieter_hintjens> “this is what you can do” is not useful but “this is how you solve problem X” is useful
[Sunday 05 September 2010] [14:09:10] <pieter_hintjens> fan-in to single server... maybe for collecting statistics
[Sunday 05 September 2010] [14:09:18] <pieter_hintjens> though why not use PUSH rather than PUB...?
[Sunday 05 September 2010] [14:30:30] <cremes> pieter_hintjens: fyi, there are some comments on that igvita article that probably need a response
[Sunday 05 September 2010] [14:30:42] <pieter_hintjens> cremes: am actually doing that right now
[Sunday 05 September 2010] [14:30:57] <cremes> i would respond if i knew anything about boost::asio or sc-whatever
[Sunday 05 September 2010] [14:31:00] <cremes> cool
[Sunday 05 September 2010] [14:31:00] <pieter_hintjens> though I doubt they “need” a response except “download it and try it”
[Sunday 05 September 2010] [14:31:07] <cremes> ha
[Sunday 05 September 2010] [14:31:21] <pieter_hintjens> i’m reminded of the virgin declaring why sex is so pointless and messy
[Sunday 05 September 2010] [14:31:50] <cremes> it’s just a good idea to make sure comments like that don’t go unanswered especially when google turns up that article in 6 months
[Sunday 05 September 2010] [14:32:14] <pieter_hintjens> ack
[Sunday 05 September 2010] [14:32:26] <cremes> l8r
[Sunday 05 September 2010] [14:32:28] <pieter_hintjens> if you have specific questions you’d like me to pontificate on, send me the URLs
[Sunday 05 September 2010] [14:32:40] <cremes> ack :)
[Sunday 05 September 2010] [14:35:31] <sigmonsays> does zeroMQ implement AMQP?
[Sunday 05 September 2010] [14:35:38] <DasIch> sigmonsays: no
[Sunday 05 September 2010] [14:37:29] <sigmonsays> [newb] is there a doc that describes what protocols it uses? I mean i’m sure the same concepts exist that AMQP implements..
[Sunday 05 September 2010] [14:41:54] <pieter_hintjens> sigmonsays: rfc.zeromq.org but it’s out of date
[Sunday 05 September 2010] [14:42:09] <pieter_hintjens> this is one thing i have wanted to make for a while, a correct spec of the protocol
[Sunday 05 September 2010] [14:42:23] <pieter_hintjens> but it is very different to AMQP
[Sunday 05 September 2010] [14:42:56] <pieter_hintjens> AMQP has stupid clients talking to a smart broker
[Sunday 05 September 2010] [14:43:01] <pieter_hintjens> 0MQ has smart clients talking to each other
[Sunday 05 September 2010] [14:43:11] <pieter_hintjens> it means the wire level protocol can be a lot simpler
[Sunday 05 September 2010] [14:50:00] <zedas> jond: i do. it uses the coroutines and zmq_poll
[Sunday 05 September 2010] [14:52:38] <sigmonsays> So, if persistence were required, would that be something you’d implement on the sending side? How does the “broker” play a role if clients talk directly to each other? I’m having a hard time finding a doc that demonstrates a traditional setup
[Sunday 05 September 2010] [14:55:05] <sigmonsays> Thanks, will do
[Sunday 05 September 2010] [14:55:32] <sustrik> in short, there are so called “devices” which are basically “mini-brokers”; it you need a full fledged broker, there’s a project that allows you to plug rabbitmq into 0MQ network
[Sunday 05 September 2010] [14:56:32] <sigmonsays> Yah, I was scanning the amq C API to see what it implements. Quite sexy API =P
[Sunday 05 September 2010] [15:02:55] <sigmonsays> Is SCTP a supported transport?
[Sunday 05 September 2010] [15:03:18] <sustrik> it would be nice if it was, but no, it isn’t yet
[Sunday 05 September 2010] [15:04:39] <sustrik> actually, it used to be implemented in old versions of 0MQ but people weren’t much interested :|
[Sunday 05 September 2010] [15:05:20] <sustrik> it seems there’s still some kind of fear of using anything other than TCP
[Sunday 05 September 2010] [15:05:37] <pieter_hintjens> I proposed to the IETF that they use SCTP as the basis for WebSockets
[Sunday 05 September 2010] [15:05:42] <pieter_hintjens> the response was interesting
[Sunday 05 September 2010] [15:05:50] * sustrik is curious
[Sunday 05 September 2010] [15:06:06] <pieter_hintjens> something along the lines of “it’s not present on most boxes and therefore will never be used by anyone at all”
[Sunday 05 September 2010] [15:06:27] <sustrik> hm, right, it not on win boxes
[Sunday 05 September 2010] [15:06:31] <pieter_hintjens> yes indeed
[Sunday 05 September 2010] [15:06:43] <pieter_hintjens> end of story, pretty much
[Sunday 05 September 2010] [15:06:52] <Zao> Nor on most FreeBSD boxen, as it conflicts with things like VIMAGE, if you’re insane enough to want that.
[Sunday 05 September 2010] [15:07:38] <pieter_hintjens> i can’t login for some weird reason
[Sunday 05 September 2010] [15:09:14] <sustrik> hm, win64 comment
[Sunday 05 September 2010] [15:09:24] <sustrik> does anyone have a win64 installation here?
[Sunday 05 September 2010] [15:09:38] <sustrik> it should be easy to fix
[Sunday 05 September 2010] [15:09:45] <sustrik> if it doesn’t work
[Sunday 05 September 2010] [15:10:25] <pieter_hintjens> sorry, my virtual box is 32-bit...
[Sunday 05 September 2010] [15:10:46] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [15:12:40] <jond> ph: the fan-in question was asked by twcc on irc about 20:00-21:00 last night. they wanted multiple publisher to single server.
[Sunday 05 September 2010] [15:13:10] <pieter_hintjens> jond: do you recall the use case?
[Sunday 05 September 2010] [15:13:58] <pieter_hintjens> “anyone have a drill for me?”
[Sunday 05 September 2010] [15:14:01] <pieter_hintjens> “why?”
[Sunday 05 September 2010] [15:14:10] <pieter_hintjens> “i have a headache that won’t go away”
[Sunday 05 September 2010] [15:14:15] <jond> ph: shall i mail the irc, I didnt answer as I wasnt around; steve-o was
[Sunday 05 September 2010] [15:14:29] <Zao> sustrik: Does he refer to PGM being hard/impossible to build?
[Sunday 05 September 2010] [15:14:34] <jond> ph: you considering trepanation then?
[Sunday 05 September 2010] [15:14:40] <pieter_hintjens> well, exactly
[Sunday 05 September 2010] [15:14:54] <pieter_hintjens> before answering people’s questions about “how to do X”, ask them “why”...
[Sunday 05 September 2010] [15:14:57] <sustrik> zao: who are you referring to?
[Sunday 05 September 2010] [15:15:00] <pieter_hintjens> i can see the irc log, will check
[Sunday 05 September 2010] [15:15:32] <Zao> sustrik: I thought that by “win64” comment you referred to the lad talking about 64-bit windows in the reddit thread?
[Sunday 05 September 2010] [15:16:02] <pieter_hintjens> jond: bah, simple PUSH->PULL pattern would do it IMO
[Sunday 05 September 2010] [15:16:22] <sustrik> ah, yes, maybe he was referring to pgm rather than 0mq as such
[Sunday 05 September 2010] [15:16:32] <sustrik> i don’t have win32 box, cannot check
[Sunday 05 September 2010] [15:16:35] <sustrik> win64*
[Sunday 05 September 2010] [15:16:36] <jond> ph: IIRC correctly he wanted intermittent/transient clients sending no-ack-needed messages to single server. it was pretty vague
[Sunday 05 September 2010] [15:16:58] <Zao> sustrik: I’ve got a 64-bit Windows box, but I don’t trust mq on it at all.
[Sunday 05 September 2010] [15:17:10] <sustrik> Zao: why so?
[Sunday 05 September 2010] [15:17:13] <sustrik> does it fail?
[Sunday 05 September 2010] [15:17:40] <pieter_hintjens> jond: i just read it twice, he wants a push/pull fanin afaics
[Sunday 05 September 2010] [15:17:50] <pieter_hintjens> it’s covered in chapter 1 of the guide even
[Sunday 05 September 2010] [15:18:00] <pieter_hintjens> should be a standard answer to people who ask for help
[Sunday 05 September 2010] [15:18:14] <pieter_hintjens> “if you’ve not read ch1 & ch2 of the guide, do so and then come back”
[Sunday 05 September 2010] [15:18:15] <Zao> sustrik: I set up a PUB and bind. I set up another thread with a SUB and connect. I subscribe to all topics. I then send lots of short messages which the SUB gets, until after a few dozen, it just blocks in zmq::poll.
[Sunday 05 September 2010] [15:18:27] <Zao> sustrik: I’ve got to boil it down to a test case before I report it on the issue tracker.
[Sunday 05 September 2010] [15:18:36] <Zao> As well as testing it with another compiler.
[Sunday 05 September 2010] [15:18:50] <sustrik> Zao: great, thanks
[Sunday 05 September 2010] [15:19:00] <Zao> This is 32-bit VC10, by the way.
[Sunday 05 September 2010] [15:19:30] <sustrik> ack
[Sunday 05 September 2010] [15:21:18] <jond> ph: yes, but you can also do a sub that binds and a pub that connects. There’s more than one way at present, which is good.
[Sunday 05 September 2010] [15:21:59] <sustrik> ph: centralised logging scanrio?
[Sunday 05 September 2010] [15:22:56] Quit andrewvc has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [15:25:54] <sigmonsays> Is the concept of headers up to the user to implement or does something like “routing-key” exist? ideally I wouldn’t want to subscribe based on the serialized data, but the headers accompanying it.
[Sunday 05 September 2010] [15:26:38] <sigmonsays> ie, ZMQ_SUBSCRIBE example from the user guide
[Sunday 05 September 2010] [15:27:28] <Zao> As far as I understand it, the topic is the prefix part of the data.
[Sunday 05 September 2010] [15:28:21] <sigmonsays> Well I wouldn’t want to unpack a serialized message to inspect it is where i’m getting
[Sunday 05 September 2010] [15:28:42] <sigmonsays> I suspect I would prefix the compress data with HTTP like headers, but i thought i’m missing something..
[Sunday 05 September 2010] [15:29:16] <sustrik> you can place your topic into one message part, data to another
[Sunday 05 September 2010] [15:29:34] <sustrik> the matching is always done on the first message part
[Sunday 05 September 2010] [15:29:44] <sigmonsays> that’s what I was missing. Thanks
[Sunday 05 September 2010] [15:31:23] <jond> sustrik: i think logging would be scenario for that use of pub/sub
[Sunday 05 September 2010] [15:32:06] <sustrik> jond: that’s what ph asked for, no?
[Sunday 05 September 2010] [15:33:58] <sigmonsays> Is there anything to facilitate message persistence? I am thinking of a WAN setup where aggregation points exist in each pop. Is that where a queue would fit in?
[Sunday 05 September 2010] [15:34:43] <DasIch> fyi logbook (a python library for logging) does now provide support for a zeromq handler/subscriber
[Sunday 05 September 2010] [15:35:08] <sustrik> you want messages just to be offloaded to disk, or do you want to survive crash/shutdown?
[Sunday 05 September 2010] [15:36:41] <sigmonsays> sustrik, good question, You mean HA replication versus disk persistence? I just want the message to be delivered and not lost/corrupted/etc.
[Sunday 05 September 2010] [15:37:01] <sustrik> i mean swapping vs. storage
[Sunday 05 September 2010] [15:37:56] <sigmonsays> Well swap dies when the box does. So that’s not ideal
[Sunday 05 September 2010] [15:38:40] <sustrik> then you have to put something that persists messages into the network
[Sunday 05 September 2010] [15:38:42] <jond> sustrik: i think so. you could probably do a really nice demo with tokyo tyrant which is the reverse index lookup thing of tokyo cabinet. especially if log messages are text
[Sunday 05 September 2010] [15:39:12] <sustrik> i’ve laready meantioned that there’s a project that allows you to use RabbitMQ broker as a node in 0mq network
[Sunday 05 September 2010] [15:39:14] <pieter_hintjens> jond: sorry, was writing a post on reddit
[Sunday 05 September 2010] [15:39:32] <pieter_hintjens> sustrik: centralized logging... yes, perhaps
[Sunday 05 September 2010] [15:39:48] <pieter_hintjens> why is that not a push/pull then?
[Sunday 05 September 2010] [15:40:12] <pieter_hintjens> i mean, pubsub is a natural pattern with subscribers that select data
[Sunday 05 September 2010] [15:40:19] <pieter_hintjens> centralized logging is a sink pattern
[Sunday 05 September 2010] [15:40:36] <pieter_hintjens> i don’t like using knives as screwdrivers, though it works
[Sunday 05 September 2010] [15:40:46] <sustrik> think of the case you would add another application listening to log feed
[Sunday 05 September 2010] [15:40:47] <sigmonsays> sustrik, I see, rabbitmq broker seems like overkill for persistence. I could see using tyrant/cabinet – Was wondering if any facilities were already provided. Still newb here.
[Sunday 05 September 2010] [15:40:55] <sustrik> would you want the logs to be load balanced?
[Sunday 05 September 2010] [15:41:04] <sustrik> i don’t think so
[Sunday 05 September 2010] [15:41:06] <pieter_hintjens> sustrik: then every single app needs to connect to TWO sinks
[Sunday 05 September 2010] [15:41:08] <pieter_hintjens> that’s insane
[Sunday 05 September 2010] [15:41:22] <pieter_hintjens> or you use a broker/forwarder
[Sunday 05 September 2010] [15:41:36] <sustrik> but not streamer
[Sunday 05 September 2010] [15:41:37] <pieter_hintjens> but then you don’t connect pubs to subs
[Sunday 05 September 2010] [15:41:42] <pieter_hintjens> no, not streamer
[Sunday 05 September 2010] [15:41:46] <pieter_hintjens> broker, in that case
[Sunday 05 September 2010] [15:41:47] <sustrik> => it’s pub/sub
[Sunday 05 September 2010] [15:42:03] <pieter_hintjens> why not push=>pull?
[Sunday 05 September 2010] [15:42:15] <sustrik> because push/pull load balances
[Sunday 05 September 2010] [15:42:19] <sustrik> rather than distributes
[Sunday 05 September 2010] [15:42:22] <pieter_hintjens> yeah, if you want multiple sinks
[Sunday 05 September 2010] [15:42:30] <sustrik> right
[Sunday 05 September 2010] [15:42:31] <pieter_hintjens> you’d do a push-pull to the broker and then pub-sub to all sinks
[Sunday 05 September 2010] [15:42:38] Quit gebi has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [15:42:44] <pieter_hintjens> still not a use case for pub->sub connections
[Sunday 05 September 2010] [15:42:49] <sustrik> you can do that, but why?
[Sunday 05 September 2010] [15:43:00] <pieter_hintjens> because it’s a sink
[Sunday 05 September 2010] [15:43:03] <pieter_hintjens> not a subscriber
[Sunday 05 September 2010] [15:43:16] <pieter_hintjens> ... message queuing semantics work better
[Sunday 05 September 2010] [15:43:26] <pieter_hintjens> i don’t want to lose log messages if broker isn’t there
[Sunday 05 September 2010] [15:43:33] <pieter_hintjens> i don’t want subscriptions
[Sunday 05 September 2010] [15:43:42] <pieter_hintjens> i want everything from all my upstream nodes
[Sunday 05 September 2010] [15:43:44] <pieter_hintjens> it’s a sink
[Sunday 05 September 2010] [15:43:52] <pieter_hintjens> pubsub is just not accurate...
[Sunday 05 September 2010] [15:44:07] <pieter_hintjens> we do IMO need a new socket type
[Sunday 05 September 2010] [15:44:12] <pieter_hintjens> which is the bus
[Sunday 05 September 2010] [15:44:19] <sustrik> shrug, it depends on you, if you believe that pipelining is the right pattern for message logs go for it
[Sunday 05 September 2010] [15:44:22] <pieter_hintjens> ... as we discussed before wrt error messages internally
[Sunday 05 September 2010] [15:44:38] <pieter_hintjens> it’s not the right pattern, it’s part of the pattern
[Sunday 05 September 2010] [15:44:43] <sustrik> pub/sub is a bus
[Sunday 05 September 2010] [15:44:48] <jond> ph+sustrik: what if we said ‘audit’ rather than logging. centralised audit of all messages?
[Sunday 05 September 2010] [15:45:11] <pieter_hintjens> audit, yes, nothing can be lost
[Sunday 05 September 2010] [15:45:29] <pieter_hintjens> pubsub is a stream with no guaranteed start
[Sunday 05 September 2010] [15:45:38] <sustrik> then yes, you want something reliable, like req/rep or pipeline
[Sunday 05 September 2010] [15:45:43] Part RunningMike has left this channel.
[Sunday 05 September 2010] [15:45:54] <sustrik> that’s what most logs are
[Sunday 05 September 2010] [15:45:56] <pieter_hintjens> yes
[Sunday 05 September 2010] [15:45:58] <sustrik> audit is different
[Sunday 05 September 2010] [15:46:04] <pieter_hintjens> logs are audits
[Sunday 05 September 2010] [15:46:12] <Zao> Oh joy. Even more fun behaviour sneaks up. zmq::poll(NULL, 0, t) asserts as winsock select craps itself on maxfd=~0.
[Sunday 05 September 2010] [15:46:14] <pieter_hintjens> losing a single message can be fatal
[Sunday 05 September 2010] [15:46:31] <pieter_hintjens> Zao: FD_SETSIZE?
[Sunday 05 September 2010] [15:47:01] Quit mato has left this server (Ping timeout: 260 seconds).
[Sunday 05 September 2010] [15:47:03] <pieter_hintjens> what is ~0...?
[Sunday 05 September 2010] [15:47:07] <sustrik> nope, zero-sized pollset
[Sunday 05 September 2010] [15:47:18] Quit keffo has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [15:47:23] Quit andrewvc has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [15:47:24] <sustrik> win32 select obviously don’t handle it well
[Sunday 05 September 2010] [15:47:36] * pieter_hintjens vaguely remembers something about that
[Sunday 05 September 2010] [15:47:38] <Zao> pieter_hintjens: maxfd is initialized to zmq::retired_fd and never gets assigned to anything else, as itemcount is 0.
[Sunday 05 September 2010] [15:47:58] <sustrik> Zao: can you report it please?
[Sunday 05 September 2010] [15:47:59] <Zao> winsock whines about invalid parameter, and the only one looking shady is the maxfd.
[Sunday 05 September 2010] [15:48:06] <Zao> sustrik: Yeah.
[Sunday 05 September 2010] [15:48:10] <sustrik> thanks
[Sunday 05 September 2010] [15:51:58] <pieter_hintjens> it looks to me that logging is a good usecase for a broker in fact
[Sunday 05 September 2010] [15:52:09] <pieter_hintjens> i’ll add this to the list of examples for the guide
[Sunday 05 September 2010] [15:55:39] <pieter_hintjens> sustrik: would it be fair to generalize that any N-to-N architecture needs a device in the middle?
[Sunday 05 September 2010] [15:55:49] <sustrik> definitely
[Sunday 05 September 2010] [15:55:53] <pieter_hintjens> except multicast, perhaps
[Sunday 05 September 2010] [15:55:55] <pieter_hintjens> ok
[Sunday 05 September 2010] [15:56:09] <sustrik> even with multicast you need a network switch in the middle
[Sunday 05 September 2010] [15:56:15] <pieter_hintjens> yes!
[Sunday 05 September 2010] [15:56:21] <sustrik> which is actaully a HW forwarder device
[Sunday 05 September 2010] [15:56:25] <pieter_hintjens> this is worth stating explicitly, it’s not clear initially why we need devices
[Sunday 05 September 2010] [15:56:32] <pieter_hintjens> i shall do this
[Sunday 05 September 2010] [15:56:37] <sustrik> ok
[Sunday 05 September 2010] [15:57:39] <Zao> Interesting. My testcase code doesn’t break.
[Sunday 05 September 2010] [15:57:42] <Zao> That’s annoying :D
[Sunday 05 September 2010] [15:57:58] <sustrik> you mean the poll bug?
[Sunday 05 September 2010] [15:58:14] <sustrik> it’s obvious what’s going on there, no need for test case
[Sunday 05 September 2010] [15:58:19] <sustrik> just report it
[Sunday 05 September 2010] [16:00:39] <Zao> sustrik: No, my other bug where poll on a SUB socket blocks even though the PUB peer keeps spamming it.
[Sunday 05 September 2010] [16:00:51] <Zao> sustrik: The one I’m trying to drum up a test case for, which doesn’t depend on D3D11.
[Sunday 05 September 2010] [16:01:00] <sustrik> ah
[Sunday 05 September 2010] [16:02:16] <pieter_hintjens> sustrik: you realize we’re totally screwed?
[Sunday 05 September 2010] [16:02:33] <sustrik> why so?
[Sunday 05 September 2010] [16:02:48] <pieter_hintjens> lol :-)
[Sunday 05 September 2010] [16:02:59] <pieter_hintjens> 10pm on a Sunday night and this is the most fun place to be
[Sunday 05 September 2010] [16:03:06] <sustrik> good god
[Sunday 05 September 2010] [16:03:09] <pieter_hintjens> yeah
[Sunday 05 September 2010] [16:03:20] <pieter_hintjens> and 0MQ is still under the radar...
[Sunday 05 September 2010] [16:03:47] <sustrik> i’ve been at a party, but returned to where fun is
[Sunday 05 September 2010] [16:03:58] <pieter_hintjens> splutter...!
[Sunday 05 September 2010] [16:04:02] <sustrik> :)
[Sunday 05 September 2010] [16:04:59] <pieter_hintjens> btw i added two concepts in the latest guide
[Sunday 05 September 2010] [16:05:17] <pieter_hintjens> 1. “0MQ string” -> length-delimited string with no trailing null
[Sunday 05 September 2010] [16:05:34] <pieter_hintjens> all the examples now correctly read/write strings in that format when they use strings
[Sunday 05 September 2010] [16:06:40] <sustrik> ok
[Sunday 05 September 2010] [16:08:38] <Zao> Hrm. Docs say that it should completely ignore the maxfd parameter.
[Sunday 05 September 2010] [16:11:54] Join snerd has joined this channel (
~motk@fedora/robk).
[Sunday 05 September 2010] [16:12:11] <sustrik> Zao: i think it does
[Sunday 05 September 2010] [16:12:25] <sustrik> Win32 fd_set is not a real fd_set
[Sunday 05 September 2010] [16:12:37] <sustrik> instead it’s a vector of SOCKETs
[Sunday 05 September 2010] [16:12:56] <sustrik> it’s vector of size 0 which causes trouble imo
[Sunday 05 September 2010] [16:16:27] Quit pieter_hintjens has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:16:27] Quit
snerd_ has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:16:27] Quit ivan has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:18:53] <sigmonsays> How does using (e)pgm allows multicast to be used reliably?
[Sunday 05 September 2010] [16:19:10] <sustrik> check RFC3208
[Sunday 05 September 2010] [16:19:18] <sustrik> there’s a resend mechanism
[Sunday 05 September 2010] [16:19:57] <sigmonsays> rgr
[Sunday 05 September 2010] [16:22:39] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [16:38:14] Part pieter_hintjens has left this channel.
[Sunday 05 September 2010] [18:11:18] Quit jsimmons has left this server (Client Quit).
[Sunday 05 September 2010] [18:48:26] Quit DasIch has left this server (Read error: Operation timed out).
[Sunday 05 September 2010] [18:49:41] Quit keffo has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [19:32:05] Quit andrewvc has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [19:38:27] Part andrewvc has left this channel.
[Sunday 05 September 2010] [19:48:22] Quit keffo has left this server (Ping timeout: 255 seconds).
[Sunday 05 September 2010] [20:36:56] Quit DasIch has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [22:18:03] <mikejs> there’s no INSTALL file in the 2.0.9 tarball, despite it being referenced in the README
[Sunday 05 September 2010] [23:14:49] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 05 September 2010] [23:18:35] Quit iFire has left this server (Remote host closed the connection).
[Sunday 05 September 2010] [23:52:10] Quit sustrik has left this server (Quit: Leaving).
[Monday 06 September 2010] [01:10:36] <mato> mikejs: good catch, my fault for not testing one setting properly, i’ll fix the tarballs
[Monday 06 September 2010] [01:14:31] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [01:59:20] Quit AndrewBC has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [02:04:43] Quit bobs has left this server (Client Quit).
[Monday 06 September 2010] [02:11:43] Part bobs has left this channel.
[Monday 06 September 2010] [02:15:30] Part bobs has left this channel.
[Monday 06 September 2010] [02:35:04] Part bobs has left this channel.
[Monday 06 September 2010] [03:18:19] Part bobs has left this channel.
[Monday 06 September 2010] [03:55:50] Quit mcxx has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [04:16:08] <BooTheHamster> hi
[Monday 06 September 2010] [04:17:20] <mato> hi BooTheHamster
[Monday 06 September 2010] [04:17:26] <BooTheHamster> can I connect ZMQ socket to zero port number, e.g. “
tcp://localhost:0”?
[Monday 06 September 2010] [04:17:55] <mato> 0 is not a legal port number, so no :-)
[Monday 06 September 2010] [04:18:34] <BooTheHamster> ZMQ don’t get a random port no?
[Monday 06 September 2010] [04:18:47] <mato> ah, you want dynamically assigned ports on socket?
[Monday 06 September 2010] [04:18:52] <BooTheHamster> yes
[Monday 06 September 2010] [04:18:54] <mato> that’s a different question :)
[Monday 06 September 2010] [04:19:08] <BooTheHamster> sorry for my english :(
[Monday 06 September 2010] [04:19:12] <mato> no problem
[Monday 06 September 2010] [04:19:33] <mato> for dynamically assigned ports, that’s not possible at the moment but you can emulate it
[Monday 06 September 2010] [04:19:50] <mato> on the bind side, you’ll want to do a loop and try to bind to a random port in a portrange
[Monday 06 September 2010] [04:20:49] <mato> lines 588 onwards, “bind_to_random_port()”
[Monday 06 September 2010] [04:21:02] <mato> you’ll then want to save the port number you bound to, and connect to that
[Monday 06 September 2010] [04:21:16] <mato> it’s a hack, I hope to resolve this properly in 2.1.x
[Monday 06 September 2010] [04:21:21] <mato> but that is still some way away
[Monday 06 September 2010] [04:21:55] <BooTheHamster> great thanx, mato :)
[Monday 06 September 2010] [04:22:12] <mato> BooTheHamster: you’re welcome
[Monday 06 September 2010] [04:23:22] Quit BooTheHamster has left this server (Quit: Page closed).
[Monday 06 September 2010] [04:31:06] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [04:32:56] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [05:15:35] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [05:18:33] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 06 September 2010] [05:20:08] Quit rbraley has left this server (Ping timeout: 245 seconds).
[Monday 06 September 2010] [05:23:41] <Tasser> no tcl binding for 2.0 yet?
[Monday 06 September 2010] [06:17:38] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [07:44:01] <BooTheHamster> Hi. Can I use telnet (or something else) to debugging my application with ZMQ_REP socket?
[Monday 06 September 2010] [07:44:28] <keffo> I suggest a debugger.
[Monday 06 September 2010] [07:47:51] <BooTheHamster> I mean that I develop a server application, but don’t have a client yet. Can I sent a messages to opened ZMQ socket?
[Monday 06 September 2010] [07:48:21] <guido_g> write a simple client first then
[Monday 06 September 2010] [07:50:43] <BooTheHamster> thanx
[Monday 06 September 2010] [07:51:40] <guido_g> there is even simpler stuff in the tests directory on master
[Monday 06 September 2010] [07:52:48] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Monday 06 September 2010] [07:52:56] <BooTheHamster> thanx, guido_g.
[Monday 06 September 2010] [08:14:03] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [09:04:47] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [09:28:07] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [09:42:24] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [09:44:03] Quit iFire has left this server (Changing host).
[Monday 06 September 2010] [10:08:03] Nick
DasIch_ is now known as DasIch.
[Monday 06 September 2010] [11:28:02] <pieterh> “This sounds like just a big marketing gimmic,”
[Monday 06 September 2010] [11:28:52] <guido_g> reddit? yes...
[Monday 06 September 2010] [11:28:59] <guido_g> ]:->
[Monday 06 September 2010] [11:31:14] Quit plq has left this server (Ping timeout: 264 seconds).
[Monday 06 September 2010] [11:32:01] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Monday 06 September 2010] [12:27:38] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Monday 06 September 2010] [12:30:26] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 06 September 2010] [13:06:45] Quit plq has left this server (Ping timeout: 258 seconds).
[Monday 06 September 2010] [13:19:01] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [13:19:41] Quit xla has left this server (Ping timeout: 260 seconds).
[Monday 06 September 2010] [13:31:40] <sbahra> pieterh, in all honesty, I first heard about ZeroMQ from the multithreading magic article (which states things like “To scale further, or developer must switch to 100% lock-free algorithms for data sharing. He is now into the realm of black magic.”), and it made it seem like ZeroMQ was a marketing gimmick. :-P
[Monday 06 September 2010] [13:32:13] <sbahra> Now, we’re seriously looking at using ZeroMQ for the product I’m involved with. :-P
[Monday 06 September 2010] [13:32:17] <pieterh> sbahra :-)
[Monday 06 September 2010] [13:32:57] <pieterh> can you share what that product is?
[Monday 06 September 2010] [13:33:26] <sbahra> If we use it, I’ll let you know.
[Monday 06 September 2010] [13:33:34] <sbahra> pieterh, it’s a distributed high performance MTA.
[Monday 06 September 2010] [13:33:45] <pieterh> nice!
[Monday 06 September 2010] [13:34:02] <pieterh> i’ve thought for a while that 0MQ was ideal for clustered products
[Monday 06 September 2010] [13:34:14] <sbahra> Based on my basic readings so far, it seems like it.
[Monday 06 September 2010] [13:34:21] <sbahra> We’re currently using spread.
[Monday 06 September 2010] [13:34:31] * pieterh does not know spread
[Monday 06 September 2010] [13:34:40] <sbahra> Much older than 0MQ, I think. :-)
[Monday 06 September 2010] [13:35:12] <pieterh> yeah, found that via google...
[Monday 06 September 2010] [13:35:30] <pieterh> interesting...
[Monday 06 September 2010] [13:36:16] <pieterh> biggest difference I see so far is that Spread uses blue while 0MQ uses red
[Monday 06 September 2010] [13:37:05] <sbahra> “Spread currently supports application messages upto around 100Kbytes. Currently the exact size is not exported as part of the API.”
[Monday 06 September 2010] [13:37:15] <sbahra> pieterh, it doesn’t seem like that’s the case for 0MQ. :-P
[Monday 06 September 2010] [13:37:28] <pieterh> oh... their latency is measured in msec...
[Monday 06 September 2010] [13:37:31] <sbahra> pieterh, spread is very resilient.
[Monday 06 September 2010] [13:38:16] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [13:38:51] <pieterh> hmm, it’s interesting...
[Monday 06 September 2010] [13:38:53] <sbahra> Anyways, we’ll see if we switch over to 0MQ sometime next year.
[Monday 06 September 2010] [13:39:03] <sbahra> I should do some hackery dackery doe. Bye.
[Monday 06 September 2010] [13:39:05] <pieterh> covers a lot of the ground people complain about missing in 0MQ
[Monday 06 September 2010] [13:39:10] <pieterh> cyat
[Monday 06 September 2010] [13:58:18] Join icy has joined this channel (
~icy@lighttpd/icy).
[Monday 06 September 2010] [14:06:33] Quit pieterh has left this server (Ping timeout: 258 seconds).
[Monday 06 September 2010] [14:09:09] Nick
AndrewBC_ is now known as AndrewBC.
[Monday 06 September 2010] [14:18:48] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:21:27] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:29:32] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:31:40] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 06 September 2010] [14:41:19] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:48:21] Quit pieterh has left this server (Ping timeout: 260 seconds).
[Monday 06 September 2010] [15:48:55] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [16:30:15] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 06 September 2010] [16:40:48] Quit andrewvc has left this server (Client Quit).
[Monday 06 September 2010] [16:41:49] Quit gebi has left this server (Ping timeout: 240 seconds).
*** on Mon Sep 6 16:59:51 2010
[Monday 06 September 2010] [16:59:51] Topic The channel topic is “Welcome!”.
[Monday 06 September 2010] [16:59:57] Mode Channel modes: no messages from outside, topic protection
[Monday 06 September 2010] [16:59:57] Created This channel was created on 2010-02-05 10:44.
[Monday 06 September 2010] [17:46:04] Part bobs has left this channel.
[Monday 06 September 2010] [17:47:41] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [19:03:43] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 06 September 2010] [20:36:59] Quit DasIch has left this server (Ping timeout: 272 seconds).
[Monday 06 September 2010] [21:39:53] Quit plq has left this server (Read error: Operation timed out).
[Monday 06 September 2010] [23:23:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [00:24:09] Quit keffo has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [01:27:02] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [01:48:19] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 07 September 2010] [01:54:18] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 07 September 2010] [01:54:29] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [01:57:45] <lestrrat> I’m having problems using fork() and making my parent process talk to the child processes via 0mq. I’m getting segfaults after the child exits and while the parent is in recv()
[Tuesday 07 September 2010] [01:57:50] <lestrrat> is this supposed to work?
[Tuesday 07 September 2010] [02:55:23] <pieterh> sustrik: you there?
[Tuesday 07 September 2010] [03:55:17] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Tuesday 07 September 2010] [04:18:48] Quit soren has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [04:32:49] Quit gebi has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [04:33:49] Quit xrfang has left this server (Remote host closed the connection).
[Tuesday 07 September 2010] [04:49:37] Quit plq has left this server (Read error: Operation timed out).
[Tuesday 07 September 2010] [05:07:21] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 07 September 2010] [05:20:45] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Tuesday 07 September 2010] [05:28:32] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 07 September 2010] [05:51:33] <pieterh> anyone here felt that XREP and XREQ could use better names?
[Tuesday 07 September 2010] [05:52:03] <lestrrat> I just gave a talk to my coworkers about zeromq, and yes, better_names++
[Tuesday 07 September 2010] [05:52:11] <pieterh> I was thinking of ROUTE and FORWARD
[Tuesday 07 September 2010] [05:53:24] * lestrrat doesn’t quite ring a bell
[Tuesday 07 September 2010] [05:53:58] <pieterh> XREP creates routing envelopes around incoming messages and uses these on output to route replies back to original clients
[Tuesday 07 September 2010] [05:54:09] Join plq has joined this channel (
~plq@78.189.30.218).
[Tuesday 07 September 2010] [05:54:12] <pieterh> XREQ just forwards messages in both directions without touching them
[Tuesday 07 September 2010] [05:55:08] <pieterh> XREP really looks like a router... it’s the only socket type that lets you address specific connections
[Tuesday 07 September 2010] [05:55:11] <keffo> I like ‘route’ very much, not forward so much
[Tuesday 07 September 2010] [05:55:36] <pieterh> yeah, forward wasn’t inspired
[Tuesday 07 September 2010] [05:55:51] <pieterh> it has to be a verb
[Tuesday 07 September 2010] [05:56:09] <pieterh> that says “move stuff in both directions but don’t mess with it”
[Tuesday 07 September 2010] [05:56:30] <pieterh> PORT
[Tuesday 07 September 2010] [05:56:45] <pieterh> XFER
[Tuesday 07 September 2010] [06:01:44] Quit andrewvc has left this server (Client Quit).
[Tuesday 07 September 2010] [06:11:56] <pieterh> lestrrat: does that ring a bell?
[Tuesday 07 September 2010] [06:13:11] <lestrrat> thinking
[Tuesday 07 September 2010] [06:15:38] <lestrrat> hmm. I grok ROUTE, but not the mudem part :) but I don’t have a great alternative plan either
[Tuesday 07 September 2010] [06:15:45] <lestrrat> naming is hard, eh.
[Tuesday 07 September 2010] [06:17:03] <pieterh> well, mudem is a play on modem, modulator/demodulator...
[Tuesday 07 September 2010] [06:17:09] Quit AndrewBC has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [06:17:14] <keffo> pieterh, I have a ‘route-codec’ in my code...
[Tuesday 07 September 2010] [06:17:18] <lestrrat> yeah, I know
[Tuesday 07 September 2010] [06:17:36] <pieterh> i don’t like invented words but if one has to invent them they should be expressive
[Tuesday 07 September 2010] [06:18:18] <keffo> whichi encodes/decodes routes for an xreq
[Tuesday 07 September 2010] [06:18:39] <pieterh> keffo: sounds right
[Tuesday 07 September 2010] [06:18:48] <pieterh> creates and uses envelopes, right?
[Tuesday 07 September 2010] [06:20:47] <keffo> it just handles them, contains a payload the ‘enduser’ is interested in, and also has a sendroute function
[Tuesday 07 September 2010] [06:20:48] <pieterh> well, an alternative to mudem: dispatch
[Tuesday 07 September 2010] [06:21:21] <keffo> naa, someone who implements any type of loadbalancing of messages is in essense a dispatcher, imo
[Tuesday 07 September 2010] [06:21:47] <pieterh> true
[Tuesday 07 September 2010] [06:22:06] <pieterh> it’s the combo of fanout and fanin
[Tuesday 07 September 2010] [06:23:06] <pieterh> i think “multiplex” is wrong since that suggest copying whereas its distribution
[Tuesday 07 September 2010] [06:23:40] <pieterh>
*it’s
[Tuesday 07 September 2010] [06:23:49] <lestrrat> yeah, I thought about multiplex, but it didn’t quite fit
[Tuesday 07 September 2010] [06:24:25] <pieterh> in terms of use cases, xreq is like push+pull, it ventilates and sinks at once
[Tuesday 07 September 2010] [06:25:24] <pieterh> one could create a nice pipeline pattern using just XREQ to XREQ
[Tuesday 07 September 2010] [06:27:11] <pieterh> how about... something more visual... 1TON
[Tuesday 07 September 2010] [06:27:57] <lestrrat> 1000kg!
[Tuesday 07 September 2010] [06:28:30] <pieterh> yeah
[Tuesday 07 September 2010] [06:28:41] <pieterh> 1-to-N for the pedantic of us
[Tuesday 07 September 2010] [06:31:05] <pieterh> it kind of feels more like a building block now
[Tuesday 07 September 2010] [06:42:46] <keffo> I have an issue where my worker process simply dissapears, but I cant seem to trap it
[Tuesday 07 September 2010] [06:42:57] <keffo> no exceptions, no atexits are run.. nada..
[Tuesday 07 September 2010] [06:42:58] <pieterh> what OS?
[Tuesday 07 September 2010] [06:42:59] <keffo> very annoying!
[Tuesday 07 September 2010] [06:43:01] <keffo> win7
[Tuesday 07 September 2010] [06:43:13] <pieterh> ah, that is a known problem
[Tuesday 07 September 2010] [06:43:21] <keffo> hu?
[Tuesday 07 September 2010] [06:43:23] <pieterh> the usual solution is to upgrade to Linux
[Tuesday 07 September 2010] [06:43:26] <pieterh> sorry :-)
[Tuesday 07 September 2010] [06:43:36] <keffo> caused by zmq??
[Tuesday 07 September 2010] [06:44:13] <keffo> tossing away 98% of the global userbase is hardly an upgrade btw :)
[Tuesday 07 September 2010] [06:44:14] <pieterh> I was kidding, my bad
[Tuesday 07 September 2010] [06:44:15] <pieterh> what language are you using?
[Tuesday 07 September 2010] [06:44:24] <keffo> c++, lua
[Tuesday 07 September 2010] [06:44:37] <pieterh> so you need a debug build of 0MQ IMO
[Tuesday 07 September 2010] [06:44:50] <keffo> oh it’s all debug, debugger is attached too :)
[Tuesday 07 September 2010] [06:44:58] <pieterh> aw :-(
[Tuesday 07 September 2010] [06:45:04] <keffo> gives me nothing.. I’ve tried all routes I can think of
[Tuesday 07 September 2010] [06:45:14] <keffo> abort()?
[Tuesday 07 September 2010] [06:45:20] <keffo> but why would that be called?
[Tuesday 07 September 2010] [06:45:20] <pieterh> DebugBreak() afair, then continue it in the debugger
[Tuesday 07 September 2010] [06:45:38] <pieterh> ... assertion failure?
[Tuesday 07 September 2010] [06:45:46] <keffo> nada :)
[Tuesday 07 September 2010] [06:46:12] <keffo> no asserts. no breakpoints, simply dissapears.. windows event log shows nothing
[Tuesday 07 September 2010] [06:46:16] <keffo> pussling actually
[Tuesday 07 September 2010] [06:46:21] <keffo> no exceptions are raised either
[Tuesday 07 September 2010] [06:46:40] <keffo> It’s as if the app cleanly exits, except it cant since it”s a while(true)
[Tuesday 07 September 2010] [06:46:57] <pieterh> it could exit in another thread I guess
[Tuesday 07 September 2010] [06:47:24] <pieterh> i’ve not worked on win32 for ages... maybe someone else here can be more helpful
[Tuesday 07 September 2010] [06:47:30] <keffo> does zmq ever call exit?
[Tuesday 07 September 2010] [06:47:39] <keffo> (or abort)
[Tuesday 07 September 2010] [06:47:41] * pieterh egreps the source code...
[Tuesday 07 September 2010] [06:47:48] Quit rbraley has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [06:48:04] <pieterh> nada
[Tuesday 07 September 2010] [06:48:09] <pieterh> asserts, yes
[Tuesday 07 September 2010] [06:48:43] <pieterh> 98%? keffo, 2010 is the Year of Linux
[Tuesday 07 September 2010] [06:48:51] <pieterh> it’s no more than 97.85% by now
[Tuesday 07 September 2010] [06:49:51] <keffo> if my mom could use any generic desktop linux without calling me, then I”d agree :)
[Tuesday 07 September 2010] [06:50:10] <pieterh> hah, my mum actually does use linux and has for years...
[Tuesday 07 September 2010] [06:51:02] <pieterh> but then again she’s currently asking me how to hide her IP address so she can troll Anonymous so perhaps she’s not typical...
[Tuesday 07 September 2010] [06:51:37] <pieterh> keffo: if you can make a reproducible case, and chop it down, maybe we can reproduce it on another platform
[Tuesday 07 September 2010] [06:52:18] <keffo> lord no, that would take ages :)
[Tuesday 07 September 2010] [06:52:34] <keffo> I just want to somehow detect -when- it happens, and go from there, but so far I’ve been unable to
[Tuesday 07 September 2010] [06:52:54] <pieterh> then, my friend, you might have to resort to...
[Tuesday 07 September 2010] [06:53:02] <pieterh> if really you have no other option...
[Tuesday 07 September 2010] [06:53:25] <keffo> print? =)
[Tuesday 07 September 2010] [06:53:28] <pieterh> yeah :-)
[Tuesday 07 September 2010] [06:53:38] <pieterh> don’t forget the fflush (stdout);
[Tuesday 07 September 2010] [06:54:52] <keffo> It’s remarkably reproducable though
[Tuesday 07 September 2010] [06:55:00] <pieterh> well, that’s always good
[Tuesday 07 September 2010] [06:55:03] <keffo> 4th time I calculate pi, it dissapears
[Tuesday 07 September 2010] [06:55:24] <pieterh> hopefully it remains stable as you add hundreds of prints
[Tuesday 07 September 2010] [06:55:40] <pieterh> you’re calculating pi?
[Tuesday 07 September 2010] [06:56:12] Quit mcxx has left this server (Quit: Lost terminal).
[Tuesday 07 September 2010] [06:57:24] <keffo> printing is not the problem, I generate ~250k of logs on each run :)
[Tuesday 07 September 2010] [06:57:42] <keffo> pi yeah, easy and verifiable thing to calc distributed :)
[Tuesday 07 September 2010] [06:58:04] <pieterh> is there an algo for distributed pi calculation somewhere?
[Tuesday 07 September 2010] [06:59:03] <keffo> sure
[Tuesday 07 September 2010] [06:59:08] <keffo> tons of different I guess
[Tuesday 07 September 2010] [06:59:10] <pieterh> i had an idea for a supermassive 0MQ project... lol
[Tuesday 07 September 2010] [06:59:39] <pieterh> not original but who cares...
[Tuesday 07 September 2010] [06:59:40] <keffo> as did I :)
[Tuesday 07 September 2010] [06:59:43] <keffo> for i=self.beginspan, self.endspan do
[Tuesday 07 September 2010] [06:59:43] <keffo> localpi = localpi + (1.0 / (i * 4.0 + 1.0) )
[Tuesday 07 September 2010] [06:59:43] <keffo> localpi = localpi - (1.0 / (i * 4.0 + 3.0) )
[Tuesday 07 September 2010] [07:00:15] <keffo> do that for each subspan of some arbitrary length, then sum them all up and the answer * 4 is pi :)
[Tuesday 07 September 2010] [07:00:30] <pieterh> not 42? weird...
[Tuesday 07 September 2010] [07:00:36] <keffo> hehe
[Tuesday 07 September 2010] [07:00:50] <pieterh> aight, so if we have a server somewhere that distributes workloads, and a simple 0MQ client that accepts them...
[Tuesday 07 September 2010] [07:01:52] <pieterh> has surely been done dozens of times
[Tuesday 07 September 2010] [07:02:08] <keffo> I’m doing a fairly more complicated scenario, but yeah
[Tuesday 07 September 2010] [07:05:18] <keffo> and I guess you can figure out why I want something a bit more flexibe than roundrobin :)
[Tuesday 07 September 2010] [07:06:00] <pieterh> right now I’m writing examples on how to use XREP to do routing
[Tuesday 07 September 2010] [07:06:25] <keffo> good, it was messy :)
[Tuesday 07 September 2010] [07:06:32] <pieterh> how so?
[Tuesday 07 September 2010] [07:06:40] <pieterh> you mean no documentation on the envelopes etc.?
[Tuesday 07 September 2010] [07:06:41] <keffo> just not very nicely explained
[Tuesday 07 September 2010] [07:06:47] <pieterh> right...
[Tuesday 07 September 2010] [07:11:26] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [07:13:09] <keffo> I would explain it as a stack+req...
[Tuesday 07 September 2010] [07:13:30] <keffo> push, push, push, payload, then pop,pop,pop, payload on the other side
[Tuesday 07 September 2010] [07:14:48] <keffo> btw, what happens in a queue device if a client never reconnects? will the msg linger indef.?
[Tuesday 07 September 2010] [07:21:02] <keffo> btw, what happens in a queue device if a client never reconnects? will the msg linger indef.?
[Tuesday 07 September 2010] [07:21:46] <pieterh> hmm, you mean a reply?
[Tuesday 07 September 2010] [07:21:53] <pieterh> with or without identity?
[Tuesday 07 September 2010] [07:22:19] <pieterh> this is what 0MQ/2.1 is fixing
[Tuesday 07 September 2010] [07:22:34] <pieterh> it will wait in some cases, discard in other cases
[Tuesday 07 September 2010] [07:22:41] <keffo> in general, the whole reconnect business
[Tuesday 07 September 2010] [07:22:53] <keffo> if it goes into the queue but never out, what happens to it?
[Tuesday 07 September 2010] [07:23:15] <pieterh> well, the queue is per socket, eventually
[Tuesday 07 September 2010] [07:23:52] * pieterh wishes the description of this had been documented beforehand
[Tuesday 07 September 2010] [07:24:13] <pieterh> there is not yet a proper explanation of how the 2.1 socket close semantics should work
[Tuesday 07 September 2010] [07:24:15] <pieterh> afaik
[Tuesday 07 September 2010] [07:24:29] <keffo> client(100msgs) -> queuedev -> service, then back again, except the client is gone forever..
[Tuesday 07 September 2010] [07:24:50] <pieterh> anonymous clients -> messages get thrown away
[Tuesday 07 September 2010] [07:25:06] <pieterh> client with identity -> messages persist as long as service is running
[Tuesday 07 September 2010] [07:25:25] <keffo> ok
[Tuesday 07 September 2010] [07:25:35] <pieterh> 0MQ does have the concept of a connection going away
[Tuesday 07 September 2010] [07:26:11] <pieterh> otherwise PUB sockets for example would end up with horrid resource leaks
[Tuesday 07 September 2010] [07:26:59] <keffo> I need to introduce some sort of session.. if I have (known)client A, does a bunch of test junk(like pi), but aborts prematurely, but then reconnects to start some other type of job, I dont want to receive a bunch of old pi results :)
[Tuesday 07 September 2010] [07:27:42] <keffo> and I would need to be able to tell all parties involved to dump everything related to an “old” session as well
[Tuesday 07 September 2010] [07:28:18] Quit mato has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [07:28:28] Quit plq has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [07:28:41] <pieterh> keffo: this starts to be industrial design work
[Tuesday 07 September 2010] [07:29:06] <keffo> pieterh, what do you mean?
[Tuesday 07 September 2010] [07:29:57] <pieterh> i mean, what you’re making is heavy duty...
[Tuesday 07 September 2010] [07:30:04] <keffo> oh very much :)
[Tuesday 07 September 2010] [07:30:25] <keffo> it has fried my brain on may occasions.. Tons of papers of diagrams spread all over the place :)
[Tuesday 07 September 2010] [07:30:32] <pieterh> if you have budget to throw at it, i can recommend an industrial 0MQ designer like Mato here
[Tuesday 07 September 2010] [07:31:07] <keffo> the bulk of the work is not the transport & topology though
[Tuesday 07 September 2010] [07:31:15] <keffo> although that needs to obviously be stable
[Tuesday 07 September 2010] [07:31:48] <pieterh> well, you need an infrastructure that understands ‘sessions’
[Tuesday 07 September 2010] [07:32:01] <keffo> Sure, but that”s already handled
[Tuesday 07 September 2010] [07:32:22] <pieterh> what do you still need then?
[Tuesday 07 September 2010] [07:32:32] <pieterh> apart from the thing not crashing...
[Tuesday 07 September 2010] [07:32:37] <keffo> hehe
[Tuesday 07 September 2010] [07:32:47] <keffo> lingering data trying to reconnect for one.
[Tuesday 07 September 2010] [07:33:20] <keffo> oh and more liberal means of implementing loadbalancing, but I’ve made that point already :)
[Tuesday 07 September 2010] [07:34:02] <pieterh> well, load balancing using XREP routing is pretty clear, and will be nicely explained in Ch3 of the Guide
[Tuesday 07 September 2010] [07:34:19] <keffo> will be? =)
[Tuesday 07 September 2010] [07:34:29] <pieterh> is in progress if I was not chatting here :-)
[Tuesday 07 September 2010] [07:34:39] <pieterh> anything to do with maintaining overall state is a different kettle of chicken, though
[Tuesday 07 September 2010] [07:34:45] <keffo> But I think I know what that will say by now :)
[Tuesday 07 September 2010] [07:35:05] <pieterh> hopefully, yeah
[Tuesday 07 September 2010] [07:35:41] <keffo> what I’m doing is something I’ve thought about for years though, so it should work :)
[Tuesday 07 September 2010] [07:35:55] <keffo> zmq solved a big gaping questionmark though :)
[Tuesday 07 September 2010] [07:36:43] <keffo> I might be getting a job soon though, so dev on this wll sadly be sidetracked to weekends and evenings only though :/
[Tuesday 07 September 2010] [07:37:32] <pieterh> is it open source?
[Tuesday 07 September 2010] [07:38:22] <keffo> it might be eventually!
[Tuesday 07 September 2010] [07:38:53] <keffo> would benefit the lua community I guess
[Tuesday 07 September 2010] [07:38:58] <pieterh> well... i’ve learned two relevant things here having done software for way too long
[Tuesday 07 September 2010] [07:39:04] <pieterh> a. if it’s not open source it will die
[Tuesday 07 September 2010] [07:39:17] <pieterh> b. if you don’t start as open source you can’t make it work afterwards
[Tuesday 07 September 2010] [07:40:02] <keffo> I dont agree, and I’ve never done anything other than software :)
[Tuesday 07 September 2010] [07:40:10] <pieterh> cause it’s not about building code but about building community...
[Tuesday 07 September 2010] [07:40:43] <pieterh> good luck, anyhow
[Tuesday 07 September 2010] [07:41:19] <keffo> I wasnt thinking of opensource as in leveraging resources, but simply aiding someone else, when I’m done with it :)
[Tuesday 07 September 2010] [07:41:49] <pieterh> nah, without people who helped make the code, it dies as soon as your evenings and weekends aren’t available any more
[Tuesday 07 September 2010] [07:42:34] <pieterh> it’s not about leveraging resources but about software that lives past the “free time” of its creator
[Tuesday 07 September 2010] [07:43:00] <pieterh> imho
[Tuesday 07 September 2010] [07:43:02] <keffo> Oh, it’s not free time at all, but I need to work for a bit to not starve :)
[Tuesday 07 September 2010] [07:43:33] <pieterh> starving is not pleasant, no
[Tuesday 07 September 2010] [07:44:20] <keffo> Not so much starving, but keeping girlfriend happier :)
[Tuesday 07 September 2010] [07:45:03] <keffo> when this is done, it will take me & the other dude about 2-3 weeks to produce the actual product we’ll eventually sell.. -That- part is already planned and so forth..
[Tuesday 07 September 2010] [07:45:17] <keffo> And ones that happens, there is no benefit to keep the code not opensource
[Tuesday 07 September 2010] [07:46:12] <keffo> err, once..
[Tuesday 07 September 2010] [07:46:53] <pieterh> :-) good spellng is hrad somtimes
[Tuesday 07 September 2010] [07:55:59] <keffo> um, this is odd
[Tuesday 07 September 2010] [07:56:10] <keffo> I wonder if lua might freak a little at some weird binary
[Tuesday 07 September 2010] [08:13:08] Nick
DasIch_ is now known as DasIch.
[Tuesday 07 September 2010] [08:50:26] Quit jsimmons has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [09:21:23] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Tuesday 07 September 2010] [09:51:25] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6d4ffd9 10/ (src/fq.cpp src/lb.cpp): Bug in fq_t and lb_t (when used via ZMQ_EVENTS option) fixed -
http://bit.ly/cvOPzL[Tuesday 07 September 2010] [10:12:19] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [10:43:46] Whois keffo is a user on channels: #zeromq
[Tuesday 07 September 2010] [10:43:46] Whois keffo is online via lindbohm.freenode.net (Stockholm, Sweden).
[Tuesday 07 September 2010] [10:43:46] Whois keffo has been idle for 2 hours, 47 minutes, and 36 seconds.
[Tuesday 07 September 2010] [10:43:46] Whois keffo has been online since 2010-09-07 02:18:32.
[Tuesday 07 September 2010] [10:43:46] Whois keffo is logged in as Keffo.
[Tuesday 07 September 2010] [10:43:46] Whois End of WHOIS list.
[Tuesday 07 September 2010] [11:14:02] <CIA-20> zeromq2: 03Martin Sustrik 07master * rf374431 10/ src/pipe.hpp : get rid of ‘has virtual functions but non-virtual destructor’ warnings in pipe.hpp -
http://bit.ly/9Relxm[Tuesday 07 September 2010] [11:21:12] <Tasser> cremes, it’s more about the ruby part you wrote
[Tuesday 07 September 2010] [11:21:36] <cremes> Tasser: i’m around if you have questions
[Tuesday 07 September 2010] [11:21:55] <Tasser> cremes, oh, just asking for the big picture
[Tuesday 07 September 2010] [11:22:03] <Tasser> aka what is where, how to stuff flows
[Tuesday 07 September 2010] [11:22:17] <cremes> sure...
[Tuesday 07 September 2010] [11:22:28] <Tasser> and probably write it down into your git
[Tuesday 07 September 2010] [11:22:35] <Tasser> HACKING or something like that :-)
[Tuesday 07 September 2010] [11:22:46] <cremes> whatever i write here, i’ll clean up and add to the README
[Tuesday 07 September 2010] [11:23:20] <cremes> ZM::Reactor is a thread that contains a single ZMQ context
[Tuesday 07 September 2010] [11:23:45] <cremes> from this context, you can create any kind of socket
[Tuesday 07 September 2010] [11:23:59] <cremes> (stop me if i’m not answering your question)
[Tuesday 07 September 2010] [11:24:21] <cremes> during socket creation, you pass a ruby object that will act as that socket’s handler
[Tuesday 07 September 2010] [11:24:32] <Tasser> callback?
[Tuesday 07 September 2010] [11:24:41] <cremes> the handler should provide on_attach, on_writable and on_readable methods
[Tuesday 07 September 2010] [11:25:32] <cremes> the on_attach method is called right away and lets you set things up (kind of like a constructor)
[Tuesday 07 September 2010] [11:25:47] <Tasser> so why not #new ?
[Tuesday 07 September 2010] [11:25:52] <cremes> the on_readable and on_writable methods are called when the socket is polled for those events and finds them to be true
[Tuesday 07 September 2010] [11:26:09] <cremes> explain what you mean by “not #new”?
[Tuesday 07 September 2010] [11:27:58] <Tasser> create a new instance per socket, so call #new and on that instance #on_writable, #on_readable
[Tuesday 07 September 2010] [11:29:26] <cremes> the handler instance is just a regular ruby class that implements the 3 methods i mentioned
[Tuesday 07 September 2010] [11:29:56] <cremes> it has a constructor (def initialize(
*args) nil; end) just like any other class
[Tuesday 07 September 2010] [11:30:16] <Tasser> less abstraction than EM
[Tuesday 07 September 2010] [11:30:21] <cremes> you could use one instance of a class to manage multiple sockets; look at the one-handed-ping-pong example
[Tuesday 07 September 2010] [11:30:33] <Tasser> yeah, having that one here atm
[Tuesday 07 September 2010] [11:30:36] <cremes> yeah, EM is kind of confusing with the EM::Connection stuff
[Tuesday 07 September 2010] [11:31:18] <Tasser> meh, gotta go :-(
[Tuesday 07 September 2010] [11:31:25] <cremes> sure
[Tuesday 07 September 2010] [11:31:50] <cremes> i’m usually on irc from 8am to 5pm central standard time (gmt -6, i think)
[Tuesday 07 September 2010] [11:32:08] <cremes> ping me if you have more questions or send them to the 0mq ml
[Tuesday 07 September 2010] [11:33:20] <bbigras> Is there a way to build zeromq with mingw?
[Tuesday 07 September 2010] [11:33:47] <cremes> bbigras: luislavena (rubyinstaller.org guy) has been playing with that
[Tuesday 07 September 2010] [11:34:14] <cremes> he opened an issue on github to fix a problem he encountered
[Tuesday 07 September 2010] [11:34:24] <cremes> so as far as i know he succeeded
[Tuesday 07 September 2010] [11:34:32] <bbigras> cremes: nice, thanks!
[Tuesday 07 September 2010] [11:35:16] <bbigras> cremes: Do you know if anyone had success using zeromq with Qt without having to disable Qt’s signal/slot macros?
[Tuesday 07 September 2010] [11:36:11] <cremes> i haven’t heard anything about that, so no
[Tuesday 07 September 2010] [11:36:23] <cremes> you might try asking on the 0mq ML
[Tuesday 07 September 2010] [11:38:07] <bbigras> cremes: thanks
[Tuesday 07 September 2010] [12:08:37] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Tuesday 07 September 2010] [12:11:23] Quit plq has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [12:20:59] Quit xla has left this server (Quit: leaving).
[Tuesday 07 September 2010] [12:36:25] Quit mato has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [12:43:48] Quit mato has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [12:54:09] Quit gebi has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [13:08:12] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 07 September 2010] [13:47:19] Quit keffo has left this server (Ping timeout: 252 seconds).
[Tuesday 07 September 2010] [13:50:51] Quit psino has left this server (Quit: psino).
[Tuesday 07 September 2010] [14:12:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [14:17:07] <ModusPwnens> hi
[Tuesday 07 September 2010] [14:17:50] <ModusPwnens> Is there anyone in here that has used google protobufs with zeromq? I’m wondering what kind of throughput is normal when using google protobufs
[Tuesday 07 September 2010] [14:19:03] Quit psino has left this server (Changing host).
[Tuesday 07 September 2010] [14:20:19] <cremes> ModusPwnens: i recommend you write a small benchmark that serializes/deserializes your data
[Tuesday 07 September 2010] [14:20:31] <cremes> and see what the upper limit is on your message rate
[Tuesday 07 September 2010] [14:20:48] <cremes> then 0mq will have that as its upper limit for throughput
[Tuesday 07 September 2010] [14:28:16] Quit dos000 has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [14:41:24] <ModusPwnens> yeah i did this cremes
[Tuesday 07 September 2010] [14:41:40] <ModusPwnens> I am just wondering whether or not my results are expected
[Tuesday 07 September 2010] [14:42:11] <ModusPwnens> Hmm, well actually, I benchmarked it with zeromq too
[Tuesday 07 September 2010] [14:42:30] <ModusPwnens> so it’s timing how long it takes to send messages as well
[Tuesday 07 September 2010] [14:44:04] <cremes> ModusPwnens: yeah, take 0mq out of the equation to get an upper bound
[Tuesday 07 September 2010] [14:44:17] <cremes> then you can test with 0mq to see what kind of overhead it is introducing
[Tuesday 07 September 2010] [14:57:44] <ModusPwnens> Hmm, another thing, I am having somewhat surprising results with the remote/local throughput tests
[Tuesday 07 September 2010] [14:58:27] <ModusPwnens> I am just using localhost, but I only get around 200 Mb/s throughput, which seems low to me.
[Tuesday 07 September 2010] [14:59:26] <cremes> ModusPwnens: try increasing the message body size from 50 bytes (from the example you posted last week) to something larger
[Tuesday 07 September 2010] [14:59:44] <cremes> also, note that the remote/local tests are doing a ping pong with REQ/REP sockets
[Tuesday 07 September 2010] [15:00:06] <cremes> you could see higher throughput on a PUB socket
[Tuesday 07 September 2010] [15:01:07] <ModusPwnens> Yeah, I have tried with larger message sizes. 5000 byte and 2500 count messages
[Tuesday 07 September 2010] [15:01:38] <ModusPwnens> is 200mb/s normal on localhost? I would have thought it would be much much faster
[Tuesday 07 September 2010] [15:01:43] <cremes> and what did you see?
[Tuesday 07 September 2010] [15:01:50] <cremes> with the varying message sizes...?
[Tuesday 07 September 2010] [15:02:20] <ModusPwnens> I see about 200mb/s or 5000 messages/s
[Tuesday 07 September 2010] [15:02:35] <cremes> so you see the same throughput regardless of message size?
[Tuesday 07 September 2010] [15:03:24] <ModusPwnens> not really, when the parameters are smaller I see different values
[Tuesday 07 September 2010] [15:03:41] <ModusPwnens> when i said 200 earlier i was using these paremeters
[Tuesday 07 September 2010] [15:03:56] <cremes> so what does 200 MB/s represent? the best that you see, the average or the worst?
[Tuesday 07 September 2010] [15:04:04] <ModusPwnens> about the average
[Tuesday 07 September 2010] [15:04:10] <ModusPwnens> i see 180 sometimes, sometimes 220
[Tuesday 07 September 2010] [15:04:33] <cremes> this is on windows, right?
[Tuesday 07 September 2010] [15:04:45] <ModusPwnens> Yeah. I was wondering if it would be faster on linux
[Tuesday 07 September 2010] [15:05:10] <cremes> sometimes; it’s hard to draw conclusions because this stuff is so dependent on OS and the hardware
[Tuesday 07 September 2010] [15:05:27] <ModusPwnens> Yeah. I notice that the official results on the zeromq website are ridiculously high
[Tuesday 07 September 2010] [15:05:38] <ModusPwnens> but I’m thinking that’s because they have a godly computer with 12 gigs of ram and 4 cores
[Tuesday 07 September 2010] [15:05:59] <ModusPwnens> i only have 1 core on this computer
[Tuesday 07 September 2010] [15:06:13] <ModusPwnens> as well as only 3 gigs of ram
[Tuesday 07 September 2010] [15:06:25] <icy> size of ram does not matter, speed does
[Tuesday 07 September 2010] [15:06:33] <ModusPwnens> I know that has an effect, but I’m not sure how large an effect that would be.
[Tuesday 07 September 2010] [15:06:36] <cremes> that computer is ancient if it has only 1 core; i don’t think intel has shipped a 1-core desktop cpu since around 2006
[Tuesday 07 September 2010] [15:07:09] <ModusPwnens> it’s actually relatively new
[Tuesday 07 September 2010] [15:07:13] <ModusPwnens> amd sempron m100
[Tuesday 07 September 2010] [15:07:22] <ModusPwnens> which i think has only 1 core
[Tuesday 07 September 2010] [15:08:47] <cremes> amd is behind the curve; sorry :)
[Tuesday 07 September 2010] [15:09:11] <ModusPwnens> Heh, apparently.
[Tuesday 07 September 2010] [15:09:18] <ModusPwnens> Anyways, is that sort of throughput expected?
[Tuesday 07 September 2010] [15:09:36] <cremes> again, it’s dependent upon OS and hardware
[Tuesday 07 September 2010] [15:09:49] <ModusPwnens> Hmm. So it’s at least not abnormal?
[Tuesday 07 September 2010] [15:10:10] <cremes> if you have nothing to compare it to...?
[Tuesday 07 September 2010] [15:10:43] <ModusPwnens> Ah, well there you go i guess.
[Tuesday 07 September 2010] [15:11:23] <icy> well, what cpu usage do you get while benchmarking?
[Tuesday 07 September 2010] [15:11:51] <icy> maybe it’s just really slow ram :)
[Tuesday 07 September 2010] [15:12:35] <ModusPwnens> i get 100% CPU usage
[Tuesday 07 September 2010] [15:13:24] <ModusPwnens> Does that mean the RAM is just slow?
[Tuesday 07 September 2010] [15:13:41] <Steve-o> what parameters are you using, I can compare with a single core Xeon right now
[Tuesday 07 September 2010] [15:13:46] <ModusPwnens> ok
[Tuesday 07 September 2010] [15:13:50] <ModusPwnens> for the cpu usage test I just ran
[Tuesday 07 September 2010] [15:13:57] <ModusPwnens> I used
[Tuesday 07 September 2010] [15:14:01] <ModusPwnens> 5000 byte messages
[Tuesday 07 September 2010] [15:14:08] <ModusPwnens> and 250,000 message count
[Tuesday 07 September 2010] [15:14:17] <ModusPwnens> all on localhost
[Tuesday 07 September 2010] [15:15:07] <Steve-o> I get 39,031 msgs/s and 1561 Mb/s
[Tuesday 07 September 2010] [15:15:10] <guido_g> message size: 1024 [B]
[Tuesday 07 September 2010] [15:15:10] <guido_g> message count: 100000
[Tuesday 07 September 2010] [15:15:10] <guido_g> mean throughput: 381327 [msg/s]
[Tuesday 07 September 2010] [15:15:10] <guido_g> mean throughput: 3123.831 [Mb/s]
[Tuesday 07 September 2010] [15:15:22] <guido_g> also a notebook
[Tuesday 07 September 2010] [15:16:25] <ModusPwnens> Hmm, that is definitely much higher than what I am getting.
[Tuesday 07 September 2010] [15:16:38] <ModusPwnens> how much ram do you have and what kind of processor?
[Tuesday 07 September 2010] [15:16:50] <ModusPwnens> guido that is
[Tuesday 07 September 2010] [15:16:53] <guido_g> Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz
[Tuesday 07 September 2010] [15:17:12] <guido_g> ram usage is not a problem at all
[Tuesday 07 September 2010] [15:17:35] <guido_g> it’s more a matter of cache and latency
[Tuesday 07 September 2010] [15:17:56] <ModusPwnens> Latency shouldn’t really be a problem on localhost though...right?
[Tuesday 07 September 2010] [15:18:16] <guido_g> ram latency
[Tuesday 07 September 2010] [15:18:20] <guido_g> sigh
[Tuesday 07 September 2010] [15:19:06] <icy> and bandwidth
[Tuesday 07 September 2010] [15:19:19] <ModusPwnens> Oh..sorry about that, i misunderstood..
[Tuesday 07 September 2010] [15:19:58] <guido_g> message size: 1024 [B]
[Tuesday 07 September 2010] [15:19:58] <guido_g> message count: 10000
[Tuesday 07 September 2010] [15:19:58] <guido_g> mean throughput: 98001 [msg/s]
[Tuesday 07 September 2010] [15:19:58] <guido_g> mean throughput: 802.824 [Mb/s]
[Tuesday 07 September 2010] [15:20:01] <guido_g> via lan
[Tuesday 07 September 2010] [15:20:21] <guido_g> so obviously you’re on a dog slow machine
[Tuesday 07 September 2010] [15:20:47] <icy> or windows :P
[Tuesday 07 September 2010] [15:21:15] <guido_g> hrhrhr
[Tuesday 07 September 2010] [15:22:47] <icy> nice webserver choice for zeromq.org
[Tuesday 07 September 2010] [15:23:15] <ModusPwnens> Hmm thanks. I will try to procure another computer to test this on.
[Tuesday 07 September 2010] [15:23:49] <guido_g> icy: you mean wikidot?
[Tuesday 07 September 2010] [15:24:13] <icy> I mean the lighttpd part :)
[Tuesday 07 September 2010] [15:31:34] <icy> hm local_thr does not seem to do anything on my osx box
[Tuesday 07 September 2010] [15:47:59] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [15:49:05] <cremes> icy: you need to run local_thr and remote_thr as a pair; one is the client and the other is the server
[Tuesday 07 September 2010] [15:49:16] <icy> ah right, thx
[Tuesday 07 September 2010] [15:53:30] <icy> ouf, this thing just sent me 1gb into swap
[Tuesday 07 September 2010] [15:54:15] <icy> sending 1000000 1kb messages
[Tuesday 07 September 2010] [15:54:35] <icy> I guess they get buffered in ram
[Tuesday 07 September 2010] [15:57:29] <cremes> icy: the receiver must not have pulled them off the queue fast enough
[Tuesday 07 September 2010] [15:57:43] <cremes> when i run those tests on my system, memory usage is constant (no queueing)
[Tuesday 07 September 2010] [15:58:08] <cremes> why don’t you pastie the arguments you passed to both programs so we can comment
[Tuesday 07 September 2010] [15:59:54] <icy> maybe I should start local_thr before remote_thr :)
[Tuesday 07 September 2010] [16:00:02] <cremes> try lowering the 1 million to 10 thousand and monitor the memory size of the programs
[Tuesday 07 September 2010] [16:00:22] <cremes> yeah, start order is important...
[Tuesday 07 September 2010] [16:01:08] <icy> doing that I get it to work even though it hits into swap briefly (understandable, the sender will always be faster)
[Tuesday 07 September 2010] [16:02:21] <cremes> icy: not true; this test is using REQ/REP sockets so it should only have 1 message in flight at any given time
[Tuesday 07 September 2010] [16:02:33] <cremes> one sender should not be able to get ahead of the other
[Tuesday 07 September 2010] [16:03:21] <cremes> (i was wrong in my statement from 2:57; no queueing should occur)
[Tuesday 07 September 2010] [16:03:22] <icy> I get 40mb ram usage with 100k messages
[Tuesday 07 September 2010] [16:03:45] <cremes> are you running the C programs or using the samples from another language binding?
[Tuesday 07 September 2010] [16:04:38] <icy> perf/ <- the ones in there which are C I think
[Tuesday 07 September 2010] [16:05:03] <cremes> did you modify the code at all?
[Tuesday 07 September 2010] [16:05:37] <icy> no, downloaded tarball, ./configure, make and ran the apps
[Tuesday 07 September 2010] [16:05:49] <cremes> huh... what OS?
[Tuesday 07 September 2010] [16:05:53] <icy> osx
[Tuesday 07 September 2010] [16:06:03] <cremes> 2.0.9?
[Tuesday 07 September 2010] [16:06:14] <cremes> 0mq, that is
[Tuesday 07 September 2010] [16:06:19] <icy> yea
[Tuesday 07 September 2010] [16:06:27] <Samy> I see a lot of emphasis on lock-free algorithms on the ZeroMQ website.
[Tuesday 07 September 2010] [16:06:28] <cremes> weird
[Tuesday 07 September 2010] [16:06:36] <cremes> it should not have unbounded memory growth
[Tuesday 07 September 2010] [16:06:42] <cremes> you should file a bug
[Tuesday 07 September 2010] [16:06:42] <Samy> What lock-free objects does ZeroMQ use? Where would the source-code be for them?
[Tuesday 07 September 2010] [16:07:02] <Samy> The atomics interface seemed too simple to support more complex data structures, though I wasn’t looking at the right thing.
[Tuesday 07 September 2010] [16:07:21] <cremes> Samy: check out the y_pipe stuff... i believe that is where the lock-free algorithms are used though sustrik would know better (he wrote it)
[Tuesday 07 September 2010] [16:07:43] <Samy> cremes, cool. Does sustrik IRC?
[Tuesday 07 September 2010] [16:08:05] <cremes> Samy: yes... he’s usually in channel but he isn’t here right now
[Tuesday 07 September 2010] [16:08:21] <Samy> Ok, thank you.
[Tuesday 07 September 2010] [16:08:34] <icy> the real mem does not show all the memory it uses as I’m already several hundred mb into swap at the time the screenshot was made
[Tuesday 07 September 2010] [16:08:55] <cremes> icy: that shouldn’t be; i would open a bug and describe the problem
[Tuesday 07 September 2010] [16:09:09] <cremes> make sure to include 0mq release, OS, OS release, etc
[Tuesday 07 September 2010] [16:09:19] <icy> k
[Tuesday 07 September 2010] [16:10:47] <cremes> icy: hold on a sec...
[Tuesday 07 September 2010] [16:11:07] <cremes> were you doing local_thr or local_lat as your perf test?
[Tuesday 07 September 2010] [16:14:00] <icy> thr
[Tuesday 07 September 2010] [16:16:59] <cremes> the local_thr/remote_thr examples don’t make any sense
[Tuesday 07 September 2010] [16:17:19] <cremes> the remote_thr program is using a REQ socket while the local_thr is using a SUB socket
[Tuesday 07 September 2010] [16:17:23] <cremes> the two are not compatible
[Tuesday 07 September 2010] [16:18:52] <icy> I’m totally knew to zeromq, just saw this benchmark app in the src dir and thought I’d give it a go :)
[Tuesday 07 September 2010] [16:18:55] <cremes> nm... i’m looking at the wrong stuff
[Tuesday 07 September 2010] [16:18:57] <cremes> argh...
[Tuesday 07 September 2010] [16:19:22] <cremes> okay, so remote_thr uses a PUB socket while local_thr uses a SUB
[Tuesday 07 September 2010] [16:19:24] <cremes> that is correct
[Tuesday 07 September 2010] [16:19:29] <cremes> you need to start local_thr first
[Tuesday 07 September 2010] [16:19:50] <cremes> remote_thr will slam your system by publishing as fast as possible, so there will be queueing
[Tuesday 07 September 2010] [16:20:05] <cremes> (i was thinking of the local_lat/remote_lat examples which uses different socket types)
[Tuesday 07 September 2010] [16:20:48] <icy> I did start local_thr first and even after that execited, remote_thr was allocating ram
[Tuesday 07 September 2010] [16:21:09] <icy> s/execited/exited/
[Tuesday 07 September 2010] [16:21:20] <cremes> yeah, i’m looking at it now...
[Tuesday 07 September 2010] [16:23:20] <cremes> if everything is working correctly, remote_thr should exit first
[Tuesday 07 September 2010] [16:37:31] Quit cremes has left this server (Quit: cremes).
[Tuesday 07 September 2010] [16:44:32] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Tuesday 07 September 2010] [16:45:13] Quit cremes has left this server (Client Quit).
[Tuesday 07 September 2010] [16:45:21] Part andrewvc has left this channel.
[Tuesday 07 September 2010] [16:46:33] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Tuesday 07 September 2010] [16:55:32] <dermoth> I’m wondering if there’s an easy way to monitor a queue size on a zeromq worker... It doesn’t seems like there is anything in the API
[Tuesday 07 September 2010] [16:55:42] <dermoth> on a zeromq broker I mean
[Tuesday 07 September 2010] [16:55:46] <dermoth> i.e. a device
[Tuesday 07 September 2010] [17:02:24] <ModusPwnens> whoa
[Tuesday 07 September 2010] [17:02:31] <ModusPwnens> im getting a strange error with the benchmarking tests now
[Tuesday 07 September 2010] [17:04:44] <dermoth> my concern is that the PUSH workers may send more messages than can be processed by the PULL workers, eventually filling up the queue. This would be possible if the PULL workers sync their state to disk to avoid data loss...
[Tuesday 07 September 2010] [17:12:07] <ModusPwnens> hey does zeroMQ allocate memory for messages all at once?
[Tuesday 07 September 2010] [17:12:25] Quit plq has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [17:13:35] <ModusPwnens> Or maybe it is because I am using the publish subscribe topology so it just continuously creates messages and the receiving end is not fast enough..
[Tuesday 07 September 2010] [17:33:14] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 07 September 2010] [18:23:07] <cremes> ModusPwnens: yes to the second thing you said; the publisher outpaces the subscriber
[Tuesday 07 September 2010] [18:23:34] <cremes> dermoth: no, there is no way to fetch the queue size; check the mailing list for the reasons why
[Tuesday 07 September 2010] [18:23:48] <cremes> that topic has been raised and answered a bunch of times (someone should add it to the FAQ)
[Tuesday 07 September 2010] [18:24:15] <cremes> dermoth: also, check out HWM (high water mark) settings for the PUSH sockets
[Tuesday 07 September 2010] [18:24:53] <cremes> by setting HWM, it will block when the queue hits that message level (or return EAGAIN if you try sending with ZMQ_NOBLOCK)
[Tuesday 07 September 2010] [18:26:05] <dermoth> yes, but I need to know berore i’ll ve blocking... I guess I could test the latency though
[Tuesday 07 September 2010] [18:26:34] <cremes> dermoth: what do you mean that you need to know before?
[Tuesday 07 September 2010] [18:26:44] <cremes> and what does latency have to do with it?
[Tuesday 07 September 2010] [18:27:21] <dermoth> well, if my queues start filling up at peak times I want to react before all pushers block... my primary goal in using zmq is to avoid blocking
[Tuesday 07 September 2010] [18:28:04] <cremes> ok, then use send with ZMQ_NOBLOCK and test for EAGAIN; when you get it then you know you have hit your high water mark
[Tuesday 07 September 2010] [18:28:14] <cremes> and you can take whatever action is necessary
[Tuesday 07 September 2010] [18:29:03] <dermoth> cremes, if the queue fill up on the device, then there will be some latency between the time I push to the queue and the time my worked gets the event. I can push a special message and have whichever worker gets it respond to me, then I know the latency. it it rises thern I need more workers downstream
[Tuesday 07 September 2010] [18:29:53] <cremes> ok
[Tuesday 07 September 2010] [18:30:25] <cremes> you could also have another pair of sockets where each worker tells the server/pusher that it has received a message
[Tuesday 07 September 2010] [18:30:54] <cremes> using this “out of band” communication, you could publish only those messages that can be immediately handled by a worker
[Tuesday 07 September 2010] [18:31:25] <cremes> you would have at most 1 message in someone’s queue because you would not push another one until each one had been acknowledged
[Tuesday 07 September 2010] [18:31:48] <cremes> that seems better than trying to rely on some weird latency calculation that might not be trustworthy
[Tuesday 07 September 2010] [18:31:51] <dermoth> i’ll implement XREQ/XREP for thing I need to make sure a worker is getting it... the rest is high-throghtput stuff that can suffer a small percentage loss...
[Tuesday 07 September 2010] [18:32:28] <cremes> definitely take a look at HWM
[Tuesday 07 September 2010] [18:32:36] <cremes> i think it does what you need
[Tuesday 07 September 2010] [18:33:27] <dermoth> the point is that I don’t want to block on the sending side... HWM will be useful in logging error conditions, but I should never end up hitting this limit...
[Tuesday 07 September 2010] [18:34:21] <cremes> dermoth: sorry, but your requirements don’t make sense to me
[Tuesday 07 September 2010] [18:34:59] <cremes> if you could get the queue length, you would probably prevent your pusher from sending more messages if it hit some threshold, right?
[Tuesday 07 September 2010] [18:35:08] <cremes> if so, then this is exactly what you can do with HWM
[Tuesday 07 September 2010] [18:35:27] <cremes> couple HWM with NO_BLOCK and you’ll get your “signal” that there aren’t enough workers
[Tuesday 07 September 2010] [18:35:38] <dermoth> well I don’t really control the pusher... it logs as data comes it. I need to have enough pullers to absorb the data as it comes in
[Tuesday 07 September 2010] [18:35:38] <cremes> assuming you ever hit the HWM
[Tuesday 07 September 2010] [18:36:28] <cremes> so you don’t have any control over throttling the pusher?
[Tuesday 07 September 2010] [18:37:24] <dermoth> because if the pushers block, it’s not goung to work anymore and I will loose data - the point is that the pushhers have to be non-blocking. But i’ll figure out something... maybe even measuring the rss size of the process might work.
[Tuesday 07 September 2010] [18:37:54] <cremes> dermoth: let me say it again then.... use send with NO_BLOCK and test for EAGAIN
[Tuesday 07 September 2010] [18:37:58] <cremes> this will NOT BLOCK
[Tuesday 07 September 2010] [18:38:05] <dermoth> yes I get that ;)
[Tuesday 07 September 2010] [18:38:20] <cremes> so what’s the problem then? ;)
[Tuesday 07 September 2010] [18:39:53] <dermoth> what I mean, EAGIN is bad too - I will log it & alert on it, but I want to know before that happens... If I start getting latency cpikes when I turn worn downstream workers, or during pikes, even before filling up the queues all the way up to the pushers, I want to know it. But that’s fine, I’ll work with what I have ;)
[Tuesday 07 September 2010] [18:40:16] <cremes> ok, i see
[Tuesday 07 September 2010] [18:40:49] <cremes> let us know what you come up with or add your solution to the wiki
[Tuesday 07 September 2010] [18:41:13] <dermoth> Sure. btw, it having multiple parallel devices wiorking well? i.e. if one crash will it nicely fall back to the other one? Will my XREQ/XREP suffer any latency? More that 2-3 seconds avg per requests may become problematic
[Tuesday 07 September 2010] [18:42:04] <dermoth> here actually on the ends it more like a REQ/REP socket, byt the device will be XREQ/XREP to pass & load-balance the messages
[Tuesday 07 September 2010] [18:43:32] <cremes> a device is just a fancy packaging for 2 sockets to aid with load balancing
[Tuesday 07 September 2010] [18:43:46] <cremes> if you are worried about a device crashing, then messages in flight could be lost
[Tuesday 07 September 2010] [18:43:49] <dermoth> i’ll probably be running some tests anyway... Thanks
[Tuesday 07 September 2010] [18:44:23] <cremes> if you can’t handle data loss, then you need to add some code around all of this to ack/nak messages and retry when things timeout or disappear
[Tuesday 07 September 2010] [18:44:34] <cremes> none of that is built in to 0mq; you have to build it on top
[Tuesday 07 September 2010] [18:44:57] <dermoth> yes... a srever crashing is pretty rare, what i’m concerned about it if the system will keep on working. the REQ/REP are for where I can’t handle loss and I will loose the request if the device crash with my message
[Tuesday 07 September 2010] [18:45:07] <cremes> and unless you have a really slow link or are sending very large messages, i can’t imagine it would ever take 2-3 seconds to send a message
[Tuesday 07 September 2010] [18:45:38] <cremes> if you use XREQ/XREP and do explicit acks, then you could have things continue to work
[Tuesday 07 September 2010] [18:46:12] <cremes> if you use REQ/REP sockets, they enforce a very strict send/recv/send/recv pattern, so a crashed peer would not be recoverable
[Tuesday 07 September 2010] [18:46:21] <dermoth> well, i’m thinking possible tcp timeouts, etc. I would assube ZeroMQ is able to handle well peers, but I will test it anyway. Thanks!
[Tuesday 07 September 2010] [18:46:31] <cremes> ok, good luck
[Tuesday 07 September 2010] [18:46:50] <dermoth> to handle well down peers
[Tuesday 07 September 2010] [18:47:44] <cremes> dermoth: TCP timeouts are not exposed to you via the 0mq api
[Tuesday 07 September 2010] [18:48:15] <cremes> 0mq may see a connection disappear at the tcp level, but that doesn’t mean that the socket is no good
[Tuesday 07 September 2010] [18:48:31] <cremes> because a 0mq socket can be bound or connected to multiple endpoints
[Tuesday 07 September 2010] [18:48:50] <cremes> one endpoint failure does not cause the whole 0mq socket to fail
[Tuesday 07 September 2010] [18:49:28] <cremes> the 0mq socket will continue to load balance messages to any “surviving” endpoints
[Tuesday 07 September 2010] [18:51:34] <dermoth> ok, sounds good. FWIW, what I have in mind to mointor PUSH/PULL latency is to send a special message with an IP/port to reply withm and have the workers respond with an udp packet. I can release the check as a Nagios plugin like I usually do for my monitoring scripts, but obviously the other end is up to the developer to get right
[Tuesday 07 September 2010] [18:53:32] <cremes> sounds neat
[Tuesday 07 September 2010] [19:06:05] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Tuesday 07 September 2010] [19:07:38] Quit keffo has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [19:09:50] Quit GeekGod has left this server (Quit: GeekGod).
[Tuesday 07 September 2010] [19:22:50] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [19:31:47] Quit psino has left this server (Quit: psino).
[Tuesday 07 September 2010] [20:37:05] Quit DasIch has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [21:11:23] Quit ModusPwnens has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [21:46:30] Quit gebi has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [22:25:30] Quit
halfie_ has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [23:47:58] Quit miguelito has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [02:41:08] Quit ivan has left this server (Ping timeout: 258 seconds).
[Wednesday 08 September 2010] [03:07:00] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [03:07:12] Quit
halfie_ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [04:12:25] <CIA-20> zeromq2: 03Martin Sustrik 07master * r91ea204 10/ (13 files in 2 dirs): EINTR returned from the blocking functions -
http://bit.ly/aSdLhb[Wednesday 08 September 2010] [04:29:31] <lestrrat> sustrik: almost.... s/Daisuke Make/Daisuke Maki/ ;) But yes, thanks for the EINTR fix!
[Wednesday 08 September 2010] [04:39:46] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 08 September 2010] [04:47:44] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [04:53:08] Quit jsimmons has left this server (Quit: Leaving).
[Wednesday 08 September 2010] [05:00:48] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 08 September 2010] [05:25:30] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 08 September 2010] [06:33:47] Quit gebi has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [07:15:44] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 08 September 2010] [07:44:46] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [07:58:48] <keffo> :)
[Wednesday 08 September 2010] [08:05:09] Quit __moore__ has left this server (Read error: Operation timed out).
[Wednesday 08 September 2010] [08:07:28] <pieterh> keffo: if I can figure out wtf I can do with an XREP talking to an XREP, I’ll finish this part of the Guide today
[Wednesday 08 September 2010] [08:07:44] <pieterh> turns out least-recently used is indecently easy to do
[Wednesday 08 September 2010] [08:11:00] <keffo> That would ameliorate at least part of the problem :)
[Wednesday 08 September 2010] [08:11:40] <keffo> Except not ‘successful send’ in my case, but when the answer is returned :)
[Wednesday 08 September 2010] [08:12:36] <keffo> It wouldn’t be very difficult for me to specify routing manually though, bypassing the zmq lb altogether
[Wednesday 08 September 2010] [08:12:46] <keffo> but I need to finish heartbeat system first
[Wednesday 08 September 2010] [08:13:18] <pieterh> well, here’s my take on custom LB...
[Wednesday 08 September 2010] [08:13:18] <keffo> I’m thinking pub -> sub->xreq back to source, and that would give me both route & availability?
[Wednesday 08 September 2010] [08:13:33] <pieterh> you want to use a XREP socket to do the routing
[Wednesday 08 September 2010] [08:13:41] <pieterh> you can use various algorithms to drive it
[Wednesday 08 September 2010] [08:13:52] <pieterh> depending on the socket type it talks to
[Wednesday 08 September 2010] [08:14:26] <keffo> I’m thinking a priorityqueue of all known/active nodes, then ‘nudging’ the weights based on speed/performance
[Wednesday 08 September 2010] [08:14:54] <pieterh> e.g. if your workers use req sockets, you get a nice LRU
[Wednesday 08 September 2010] [08:15:14] <pieterh> you don’t need a priority queue, just the message from the worker saying “ready”
[Wednesday 08 September 2010] [08:15:31] <keffo> hum, explain how req gives lru?
[Wednesday 08 September 2010] [08:15:43] <pieterh> well, think of HTTP long poll
[Wednesday 08 September 2010] [08:15:59] <pieterh> flow is worker says “ready”, router provides it workload
[Wednesday 08 September 2010] [08:16:09] <pieterh> workers say “ready” when they’re done...
[Wednesday 08 September 2010] [08:16:21] <pieterh> ready messages come into queue, router takes one off each time
[Wednesday 08 September 2010] [08:20:39] <keffo> uh, must draw on paper..
[Wednesday 08 September 2010] [08:21:03] <pieterh> hey, let me post this section of Ch 3, see if it makes any sense at all
[Wednesday 08 September 2010] [08:21:06] <pieterh> :-)
[Wednesday 08 September 2010] [08:21:19] <keffo> I think I need two prioqueues to be honest, one for picking the best of available set of nodes, and one for handing the most urgent tasks
[Wednesday 08 September 2010] [08:21:51] <keffo> let me know when you finish it!
[Wednesday 08 September 2010] [08:22:47] <guido_g> playing the “naming awareness” card: this isn’t REQuest/REPly anymore then, right?
[Wednesday 08 September 2010] [08:24:31] <pieterh> nope, this ain’t kansas anymore
[Wednesday 08 September 2010] [08:24:44] <pieterh> this is “off-road”...
[Wednesday 08 September 2010] [08:27:12] <pieterh> basically it’s about using identities and constructing envelopes so that XREP will route as we want to other nodes
[Wednesday 08 September 2010] [08:27:53] <pieterh> I just posted what I have...
[Wednesday 08 September 2010] [08:31:23] Quit rossij has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [08:33:13] <keffo> looks good
[Wednesday 08 September 2010] [08:35:28] <keffo> except the pub/sub ctrl+c sample code needs linebreaks
[Wednesday 08 September 2010] [08:37:57] <keffo> hum, it gets progressively difficult to read after that :)
[Wednesday 08 September 2010] [08:37:58] <pieterh> what sample is that?
[Wednesday 08 September 2010] [08:38:03] <pieterh> :-)
[Wednesday 08 September 2010] [08:38:35] <keffo> something got weird halfway down the article, it’s still embedded in the code preview
[Wednesday 08 September 2010] [08:38:45] <pieterh> hang on, refresh the page
[Wednesday 08 September 2010] [08:38:55] <pieterh> there was a fault in the text, i already fixed it
[Wednesday 08 September 2010] [08:39:16] <keffo> ignore, reloaded :)
[Wednesday 08 September 2010] [08:39:28] <keffo> lots of good food for thought there
[Wednesday 08 September 2010] [08:39:37] <guido_g> ack
[Wednesday 08 September 2010] [08:39:44] <guido_g> fig 22, the dashes are misplaced
[Wednesday 08 September 2010] [08:40:09] * keffo bbll
[Wednesday 08 September 2010] [08:40:10] <pieterh> guido_g: this is ditaa being too clever
[Wednesday 08 September 2010] [08:40:21] <pieterh> it thinks ‘-‘ in text is me trying to draw a line
[Wednesday 08 September 2010] [08:40:33] <guido_g> ouch
[Wednesday 08 September 2010] [08:40:49] <pieterh> ill try a unicode n-dash or something
[Wednesday 08 September 2010] [08:40:59] <guido_g> so no dash between single and hop and a slash for req/rep :)
[Wednesday 08 September 2010] [08:41:40] <pieterh> it’s always “request-reply”...
[Wednesday 08 September 2010] [08:42:09] <guido_g> ahh... then...
[Wednesday 08 September 2010] [08:42:40] <pieterh> np, easy to fix this in the parser
[Wednesday 08 September 2010] [08:45:52] <pieterh> bah, ditaa does not like Unicode at all, just dies
[Wednesday 08 September 2010] [08:45:53] <pieterh> well, too bad
[Wednesday 08 September 2010] [08:55:20] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [08:55:47] <guido_g> no escape mechanism?
[Wednesday 08 September 2010] [08:57:30] <pieterh> no mechanisms of any kind afaics
[Wednesday 08 September 2010] [08:57:45] <guido_g> oh
[Wednesday 08 September 2010] [08:57:50] <pieterh> which is ok, I’ll email the author
[Wednesday 08 September 2010] [09:16:50] Quit
keffo_ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [09:46:13] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47e87b7 10/ include/zmq.h : EMTHREAD error code returned to zmq.h to ensure backward compatibility -
http://bit.ly/dtznl6[Wednesday 08 September 2010] [10:07:26] <sd88g93> greetings !
[Wednesday 08 September 2010] [10:08:44] <sd88g93> when are the zmq_msg_move() zmq_msg_copy() functions meant to be used ? i mean, what situations call for them ?
[Wednesday 08 September 2010] [10:14:57] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [10:27:50] <cremes> sd88g93: whenever you send data, the library takes ownership of your zmq_msg_t and calls close() on it
[Wednesday 08 September 2010] [10:27:59] <cremes> on the recv side, you are responsible for its lifecycle
[Wednesday 08 September 2010] [10:28:35] <cremes> so you might want to make a copy (which i think does reference counting internally) so that you could send it without it being deallocated
[Wednesday 08 September 2010] [10:28:47] <cremes> the copy is cheap (ref counting)
[Wednesday 08 September 2010] [10:28:56] <cremes> i don’t really know of a use case for move
[Wednesday 08 September 2010] [10:29:14] <sd88g93> cremes: what about VSM’s, where the message is embedded right in the msg struct ?
[Wednesday 08 September 2010] [10:29:23] <cremes> i don’t know
[Wednesday 08 September 2010] [10:29:32] <sd88g93> ohok
[Wednesday 08 September 2010] [10:29:35] <cremes> i’d have to read the code to figure out what happens
[Wednesday 08 September 2010] [10:30:06] <sd88g93> well, what about if you are recieving messages of a multi part message and relaying them through inproc to another thread ?
[Wednesday 08 September 2010] [10:30:19] <sd88g93> is it alright to just recieve and send the same msg ?
[Wednesday 08 September 2010] [10:30:30] <cremes> yes
[Wednesday 08 September 2010] [10:30:48] <cremes> recv does not automatically call close on the message, only sending does
[Wednesday 08 September 2010] [10:30:51] <sd88g93> i’m recieving a multipart message by way of a req/rep socket, and want to relay them to ap ublisher
[Wednesday 08 September 2010] [10:31:32] <sd88g93> but i’m relaying them first through inproc to the main thread, and then forwarding them to the one pub skt
[Wednesday 08 September 2010] [10:31:44] <cremes> oh, so two hops?
[Wednesday 08 September 2010] [10:31:45] <sd88g93> the inproc is failing, and just trying to figure out why
[Wednesday 08 September 2010] [10:31:58] <sd88g93> yeah, 2 hops
[Wednesday 08 September 2010] [10:32:28] <cremes> well, i’m not sure... i haven’t played much with inproc transports
[Wednesday 08 September 2010] [10:32:44] <cremes> as an optimization it might be doing ref counting on the message so that it doesn’t have to copy it for real
[Wednesday 08 September 2010] [10:32:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [10:32:52] <sd88g93> actually, it would be 3 hops, once to the thread, then once to the main thread, and then the forward device forwards them to the final socket for a send
[Wednesday 08 September 2010] [10:33:04] <cremes> only count sockets as hops
[Wednesday 08 September 2010] [10:33:38] <sd88g93> yeah, so 3 hops
[Wednesday 08 September 2010] [10:34:11] <cremes> why don’t you try using copy on the message received that you are sending to the inproc socket and see if that helps (shooting in the dark here)
[Wednesday 08 September 2010] [10:34:36] <sd88g93> yeah, i was trying move
[Wednesday 08 September 2010] [10:34:44] <sd88g93> but maybe copy would be better
[Wednesday 08 September 2010] [10:34:51] <cremes> so, XREQ(tcp) -> XREP(inproc) -> PUB(tcp)
[Wednesday 08 September 2010] [10:34:53] <sd88g93> i dont need to use it after relaying it
[Wednesday 08 September 2010] [10:34:56] <cremes> copy at the XREQ step
[Wednesday 08 September 2010] [10:35:32] <sd88g93> actually, i was using PUB for the inproc
[Wednesday 08 September 2010] [10:35:39] <cremes> the main thing to recall is that whenever you send through a socket, it calls close on the message you just sent
[Wednesday 08 September 2010] [10:35:49] <cremes> you relinquish control of the message to the library whenever you send anything
[Wednesday 08 September 2010] [10:36:13] <cremes> so it sounds to me like when you send to the inproc socket, the message may be getting closed on the send side
[Wednesday 08 September 2010] [10:36:21] <cremes> this is all kind of confusing :)
[Wednesday 08 September 2010] [10:36:33] <cremes> i recommend you ping mato or sustrik
[Wednesday 08 September 2010] [10:37:19] <sd88g93> in the worker thread, it is a PUB skt, and then in the main thread i have a sub skt, so i publish from the thread via inproc, and then it gets fwd’d to the main pub skt
[Wednesday 08 September 2010] [10:37:46] <sd88g93> oh ok
[Wednesday 08 September 2010] [10:37:57] <cremes> are you using a forwarder device or did you roll your own?
[Wednesday 08 September 2010] [10:38:05] <sd88g93> fwd device
[Wednesday 08 September 2010] [10:38:14] <cremes> are you using release 2.0.9?
[Wednesday 08 September 2010] [10:38:25] <sd88g93> yeah, i just updated to 2.0.9
[Wednesday 08 September 2010] [10:38:47] <cremes> ok... i think there was a bug in the forwarder for multi-part message handling that was fixed in 2.0.9
[Wednesday 08 September 2010] [10:39:03] <cremes> that could have been your problem, but if you updated then maybe not
[Wednesday 08 September 2010] [10:39:17] <sd88g93> at first i had my own queue to queue the messages for a publisher thread, that worked, except i’m worried about when i stress test it, if pasing messages allocated on the heap, is a problem when i free them
[Wednesday 08 September 2010] [10:39:46] <sd88g93> yeah, i noticed that in the change log when i updated
[Wednesday 08 September 2010] [10:39:52] <cremes> i’d have to see code to be sure but it doesn’t sound like a problem
[Wednesday 08 September 2010] [10:40:20] <cremes> sorry i couldn’t be of more help... if you figure it out please post a solution
[Wednesday 08 September 2010] [10:40:25] Quit gebi has left this server (Ping timeout: 252 seconds).
[Wednesday 08 September 2010] [10:40:46] <sd88g93> sure, no proble, i realize its still a young project, one of the ramifications
[Wednesday 08 September 2010] [10:41:25] <sd88g93> over all, ive noticed the documentation for zeromq getting much better and more concise as time goes on
[Wednesday 08 September 2010] [10:42:10] <sd88g93> i’ll try using message copy
[Wednesday 08 September 2010] [10:44:03] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [10:44:33] <cremes> great; you might also consider posting your question (with more detail) to the ML
[Wednesday 08 September 2010] [10:52:03] <pieterh> sd88g93: in fact I don’t know what cases need zmq_msg_move and zmq_msg_copy
[Wednesday 08 September 2010] [10:53:10] <sd88g93> ok, thanks, pieterh
[Wednesday 08 September 2010] [10:53:29] <pieterh> perhaps if you want to make abstraction layers that want to grab a copy of a message, work with it, and allow the caller to continue and possibly deallocate it
[Wednesday 08 September 2010] [10:53:48] <pieterh> you certainly do not need them for normal work afaics
[Wednesday 08 September 2010] [10:53:56] <cremes> pieterh: if you want to send the same data to multiple sockets, you need to copy it
[Wednesday 08 September 2010] [10:54:02] <sd88g93> yeah, the move/copy didnt work for my situtaion
[Wednesday 08 September 2010] [10:54:13] <cremes> otherwise your first send will deallocate it out from under you, right?
[Wednesday 08 September 2010] [10:55:03] <mato> cremes: send never deallocates anything out from under you. zmq_msg_t is reference counted.
[Wednesday 08 September 2010] [10:55:35] <cremes> mato: if refcount is 1 and you send the message, it gets deallocated, right?
[Wednesday 08 September 2010] [10:55:40] <sd88g93> oh, so a call to send increments the ref counter
[Wednesday 08 September 2010] [10:55:55] <cremes> i would think it decrements the counter
[Wednesday 08 September 2010] [10:56:03] <pieterh> ... this is all invisible... you send as often as you like and don’t even know there’s a reference counter
[Wednesday 08 September 2010] [10:56:11] <sd88g93> i mean, incrememnt at first, nd then decrement when its done wiht it
[Wednesday 08 September 2010] [10:56:32] <cremes> recv, refcount + 1, send, refcount - 1
[Wednesday 08 September 2010] [10:56:39] <cremes> when 0, dealloc
[Wednesday 08 September 2010] [10:56:44] <mato> cremes: zmq_send() does not touch the refcount
[Wednesday 08 September 2010] [10:56:58] <mato> cremes: you should always do zmq_msg_close () if you want the message to go away
[Wednesday 08 September 2010] [10:56:59] <cremes> doesn’t it call zmq_msg_close?
[Wednesday 08 September 2010] [10:57:12] <mato> not unless something changed; let me check 100%
[Wednesday 08 September 2010] [10:57:16] <cremes> ok
[Wednesday 08 September 2010] [10:57:26] <cremes> i thought i saw that when i reviewed the code a few weeks ago
[Wednesday 08 September 2010] [10:57:35] * pieterh is pretty sure send() does a copy and then a close
[Wednesday 08 September 2010] [10:57:52] <mato> pieterh: no way
[Wednesday 08 September 2010] [10:58:11] <mato> what all the perf tests and examples do is send, then immediately close
[Wednesday 08 September 2010] [10:58:37] <mato> unless i’m confused, check with sustrik to get it from the horses mouth
[Wednesday 08 September 2010] [10:58:41] <pieterh> yes, that’s the calling code
[Wednesday 08 September 2010] [10:58:47] <mato> but this is what i’ve always done in my code
[Wednesday 08 September 2010] [10:58:55] <pieterh> what does the send method do internally to keep the message alive
[Wednesday 08 September 2010] [10:59:02] <pieterh> while the caller does zmq_msg_close?
[Wednesday 08 September 2010] [10:59:08] <mato> ah, sorry
[Wednesday 08 September 2010] [10:59:19] <mato> yeah, send increases the refcount by 1 obviously
[Wednesday 08 September 2010] [10:59:27] <mato> but you get a refcount of 1 already when you create the message
[Wednesday 08 September 2010] [10:59:38] <pieterh> using copy, I assume... ?
[Wednesday 08 September 2010] [10:59:44] <mato> refcount
[Wednesday 08 September 2010] [10:59:49] <mato> what copy?
[Wednesday 08 September 2010] [10:59:59] <sd88g93> oh, so it increases by one, hten when its done sending , it decreases the ref count
[Wednesday 08 September 2010] [11:00:02] <pieterh> so directly modifying message property?
[Wednesday 08 September 2010] [11:00:12] <mato> some lock-free magic with atomic_counter AFAIK
[Wednesday 08 September 2010] [11:00:14] <mato> yes
[Wednesday 08 September 2010] [11:00:31] <mato> all the docs say “zmq_msg_t is opaque”, never touch it outside of the zmq_msg_* functions
[Wednesday 08 September 2010] [11:00:34] <mato> for precisely this reason
[Wednesday 08 September 2010] [11:00:38] <sd88g93> what about with VSM’s ?
[Wednesday 08 September 2010] [11:00:57] <mato> sd88g93: it’s up to 0MQ if it actually copies the content
[Wednesday 08 September 2010] [11:01:18] <pieterh> mato: this is what zmq_msg_copy does, increment refcount
[Wednesday 08 September 2010] [11:01:52] <sd88g93> so, if you have a VSM, and you send it, and then close it, if you re-init the struct, you wipe out what send would have ?
[Wednesday 08 September 2010] [11:02:15] <mato> pieterh: i’m not sure what it actually does internally, why do you need to know?
[Wednesday 08 September 2010] [11:02:19] <sd88g93> as far as i can tell, the VSM is right inside the struct
[Wednesday 08 September 2010] [11:02:23] <mato> pieterh: anyhow, if you need to know, ask sustrik
[Wednesday 08 September 2010] [11:02:27] <pieterh> sure
[Wednesday 08 September 2010] [11:02:35] <mato> sd88g93: ask sustrik on the mailing list, i’m not an expert on what happens internally
[Wednesday 08 September 2010] [11:02:37] <pieterh> the discussion here is “when do we use zmq_msg_copy/move”
[Wednesday 08 September 2010] [11:02:39] <mato> i just know that it works :)
[Wednesday 08 September 2010] [11:02:49] <pieterh> indeed...
[Wednesday 08 September 2010] [11:03:08] <pieterh> i’m just saying, applications do not normally mess with the refcounts, ever
[Wednesday 08 September 2010] [11:03:28] <sd88g93> mato, i’m just having a problem with relaying a message i get from a REP socket over PUB skt using inproc
[Wednesday 08 September 2010] [11:03:29] <pieterh> that’s internal, and i assume that send() calls copy() to grab a reference, and close() when it’s done, to release it
[Wednesday 08 September 2010] [11:03:29] <mato> well, unless you count calling zmq_msg_close() as “messing with the refcount”...
[Wednesday 08 September 2010] [11:03:50] <pieterh> well, sure, but that’s not the semantic
[Wednesday 08 September 2010] [11:03:59] <pieterh> the semantic is “I’m finished with it”, which translates to “decrement refcount”
[Wednesday 08 September 2010] [11:04:04] <mato> yup
[Wednesday 08 September 2010] [11:04:19] * pieterh doesn’t want people to start thinking they have to copy/move messages to send them
[Wednesday 08 September 2010] [11:04:31] <pieterh> sg88g93: the problem is elsewhere
[Wednesday 08 September 2010] [11:04:45] <mato> sd88g93: don’t forget zmq_recv() destroys existing message content, if any
[Wednesday 08 September 2010] [11:05:07] <pieterh> sd88g93, do you have a minimal example?
[Wednesday 08 September 2010] [11:05:28] <sd88g93> i have some code, but so far, i have been unablet to dupliate it in a minimal example,
[Wednesday 08 September 2010] [11:05:47] <mato> sd88g93: so e.g. if you’ve got a loop, doing zmq_msg_init() then zmq_recv(), what could be happening is you recv, then send that message, but then recv back into the same message so you could lose messages
[Wednesday 08 September 2010] [11:05:55] <sd88g93> i posted to the ML one example that i thought i duplicated it , but it didnt
[Wednesday 08 September 2010] [11:06:25] <pieterh> it copies messages from one socket to another, nothing bizarre
[Wednesday 08 September 2010] [11:06:41] <pieterh> if you’re losing messages, perhaps the sockets aren’t connected properly
[Wednesday 08 September 2010] [11:07:22] <mato> sd88g93/pieterh: AFAICT zmq_msg_copy()/zmq_msg_move() are for the case where the app for some reason re-uses message “objects” (i.e. the zmq_msg_t) with a different lifecycle than the message content
[Wednesday 08 September 2010] [11:07:39] <pieterh> right
[Wednesday 08 September 2010] [11:08:06] <mato> sd88g93: what language are you using?
[Wednesday 08 September 2010] [11:08:15] <sd88g93> the c bindings
[Wednesday 08 September 2010] [11:08:57] <pieterh> sd88g93, what is the problem you are actually seeing?
[Wednesday 08 September 2010] [11:09:36] <sd88g93> that’s a minimal case, that one works, but it mirrors what i’m doing in the larger project
[Wednesday 08 September 2010] [11:10:03] <pieterh> so publisher is not sending anything out
[Wednesday 08 September 2010] [11:10:04] <sd88g93> for some reason in the larger project the messages in the main thread arent being recieved via inproc by the subscr socket
[Wednesday 08 September 2010] [11:10:15] <sd88g93> yeah, the publisher isnt sending
[Wednesday 08 September 2010] [11:10:27] <sd88g93> ive isolated it to over the inproc that its failing
[Wednesday 08 September 2010] [11:10:40] <pieterh> try changing the transport to ipc
[Wednesday 08 September 2010] [11:10:55] <sd88g93> when i hcange it to one thread , and publish directly to the bound extreneral interface, it sends fine
[Wednesday 08 September 2010] [11:11:48] <pieterh> so it’s a problem with thread coordination, connections not happening in time
[Wednesday 08 September 2010] [11:11:50] <sd88g93> what’s the difference between inproc and ipc ?
[Wednesday 08 September 2010] [11:12:08] <pieterh> well, inproc requires that you bind first, connect second
[Wednesday 08 September 2010] [11:12:15] <pieterh> ipc like tcp lets you connect first, then bind
[Wednesday 08 September 2010] [11:12:24] <sd88g93> the connections seem to be happending, ive logged it and they seem to happen, the bound channels are made before the connect happens
[Wednesday 08 September 2010] [11:12:36] <pieterh> so with inproc if the subscribers connect before the publisher binds, it won’t work
[Wednesday 08 September 2010] [11:12:42] <pieterh> IMO connect will even return an error
[Wednesday 08 September 2010] [11:13:08] <sd88g93> no, it binds first, and then connects
[Wednesday 08 September 2010] [11:13:35] <sd88g93> that shouldnt be a problem, i think you are correct in that it returns an error, ive recieved that error when that happens
[Wednesday 08 September 2010] [11:13:39] <mato> sd88g93: this is what i meant with my comment about recv’ing into the same message:
http://pastebin.com/tLumsbia[Wednesday 08 September 2010] [11:13:46] <pieterh> yeah, this is your problem I think
[Wednesday 08 September 2010] [11:13:51] <mato> sd88g93: see the difference between the correct/incorrent case
[Wednesday 08 September 2010] [11:13:57] <mato> sd88g93: just in case that’s what you’re doing...
[Wednesday 08 September 2010] [11:14:02] <pieterh> you’re starting the publishers in child threads
[Wednesday 08 September 2010] [11:14:12] <pieterh> but immediately you connect the subscribers to those ports
[Wednesday 08 September 2010] [11:14:15] <pieterh> won’t work
[Wednesday 08 September 2010] [11:14:57] <pieterh> magic solution: do two loops, one to launch pub threads, second to prepare subscribers
[Wednesday 08 September 2010] [11:15:02] <pieterh> do sleep(1) in between two loops
[Wednesday 08 September 2010] [11:15:18] <pieterh> you owe me a beer if it works
[Wednesday 08 September 2010] [11:16:27] <pieterh> here is the text from the user guide:
[Wednesday 08 September 2010] [11:16:27] <pieterh> The {{inproc}} transport has a specific limitation compared to {{ipc}} and {{tcp}}: you must do bind before connect. This is something future versions of 0MQ may fix, but for today it has some impact on the way you use inproc sockets. In the example here, we carefully bind to each endpoint before connecting to it. If you connect first, and then bind, the connect will return an error, and if you ignore that error, the recv will block.
[Wednesday 08 September 2010] [11:17:29] <sd88g93> mato: i dont think that’ sthe case, because i re init the message at the beginning of the loop
[Wednesday 08 September 2010] [11:18:44] <sd88g93> pieterh: it seeems to be binding first before connect
[Wednesday 08 September 2010] [11:18:54] <pieterh> how do you know?
[Wednesday 08 September 2010] [11:19:03] <pieterh> your example does the bind after the connect
[Wednesday 08 September 2010] [11:19:03] <sd88g93> because i print it out
[Wednesday 08 September 2010] [11:19:09] <pieterh> means nothing
[Wednesday 08 September 2010] [11:19:19] <sd88g93> oh
[Wednesday 08 September 2010] [11:19:20] <pieterh> printing from multiple threads can do weird stuff
[Wednesday 08 September 2010] [11:19:35] <pieterh> follow the printf with fflush(stdout) to be sure
[Wednesday 08 September 2010] [11:19:41] <pieterh> sd88g93, next time you have an error, if 0MQ reports an error, that’s kind of the first thing you look at
[Wednesday 08 September 2010] [11:20:16] <pieterh> “I’m using inproc and my connect fails with an error... why?”
[Wednesday 08 September 2010] [11:20:33] <sd88g93> but the connect doesnt fail ,
[Wednesday 08 September 2010] [11:20:33] <
keffo__> is the log pub in 2.0.8?
[Wednesday 08 September 2010] [11:21:27] <pieterh>
keffo__, nope
[Wednesday 08 September 2010] [11:21:33] <pieterh> sd88g93, so what fails?
[Wednesday 08 September 2010] [11:21:46] <pieterh> and could you try the ‘magic fix’ i suggested, it’s a quick test
[Wednesday 08 September 2010] [11:21:57] <sd88g93> the program doesnt throw any errors, it just doesnt publish any of the messages
[Wednesday 08 September 2010] [11:22:08] <sd88g93> yes, i ‘ll tryt that test
[Wednesday 08 September 2010] [11:22:12] <pieterh> ” i think you are correct in that it returns an error, ive recieved that error when that happens”
[Wednesday 08 September 2010] [11:22:15] <pieterh> ?
[Wednesday 08 September 2010] [11:22:35] <sd88g93> sorry, i’m just reading over what everyone has written, its alot to digest
[Wednesday 08 September 2010] [11:22:44] <pieterh> NP :-)
[Wednesday 08 September 2010] [11:22:44] <sd88g93> i’ll try that fix now, thanks,
[Wednesday 08 September 2010] [11:22:58] <pieterh> did you try it with ipc:?
[Wednesday 08 September 2010] [11:23:24] <pieterh> if that works, and inproc does not, it’s most likely the bind/connect order
[Wednesday 08 September 2010] [11:37:17] <sd88g93> no, still nothing
[Wednesday 08 September 2010] [11:37:29] <sd88g93> i’ll try it with ipc , just to be sure
[Wednesday 08 September 2010] [11:37:40] <pieterh> ack
[Wednesday 08 September 2010] [11:39:19] <pieterh> there is some questionable stuff in your example BTW
[Wednesday 08 September 2010] [11:39:32] <pieterh> like threads modifying shared static variables
[Wednesday 08 September 2010] [11:40:39] <sd88g93> yeah, i dont do that n the main program
[Wednesday 08 September 2010] [11:41:34] <sd88g93> the shared static variable is just to get a number to form the pipe number back to the main program, in the actual example i pass it in through the parameter when creating the thread
[Wednesday 08 September 2010] [11:42:18] <pieterh> hang on, do you actually have any subscribers connecting to 127.0.0.1:4000?
[Wednesday 08 September 2010] [11:42:29] <sd88g93> yes
[Wednesday 08 September 2010] [11:43:14] <pieterh> well, what I would do is just break this up into little pieces
[Wednesday 08 September 2010] [11:43:17] <sd88g93> that example does seem to work, but the same scenario, doesnt play out in the real project
[Wednesday 08 September 2010] [11:43:30] <pieterh> ah, it works in the example?
[Wednesday 08 September 2010] [11:43:53] <sd88g93> yeah, just cant duplicate it , that example was just to give you an idea of what i am doing
[Wednesday 08 September 2010] [11:44:10] <pieterh> can’t debug an idea... :-/
[Wednesday 08 September 2010] [11:44:13] <sd88g93> sorry, didnt mean to confuse
[Wednesday 08 September 2010] [11:44:34] <sd88g93> yeah, trying to isolate where its coming from
[Wednesday 08 September 2010] [11:44:50] <pieterh> so you can write your own forwarding code
[Wednesday 08 September 2010] [11:44:55] <pieterh> it is trivial
[Wednesday 08 September 2010] [11:45:02] <pieterh> then you can debug that quite easily
[Wednesday 08 September 2010] [11:45:19] <sd88g93> but when i try a loop in the main program to read what’s coming from the subscriber socket via inproc, nothing comes from it
[Wednesday 08 September 2010] [11:45:31] <pieterh> your problem maybe is that by starting an in-built device you have a black box you can’t inspect
[Wednesday 08 September 2010] [11:45:58] <pieterh> so it’s not the publisher not sending, it’s the subscriber not receiving?
[Wednesday 08 September 2010] [11:46:28] <sd88g93> the publisher isnt making it to the outer publisher socket, nothings coming out
[Wednesday 08 September 2010] [11:46:48] <sd88g93> there’s not errors from the zmq_send function,
[Wednesday 08 September 2010] [11:47:30] <sd88g93> but when i monitor the socket with tshark, there’s nothing coming out
[Wednesday 08 September 2010] [11:47:37] <pieterh> sd88g93, sorry, I can’t really help you any more until/unless you can get a minimal test case that reproduces the problem
[Wednesday 08 September 2010] [11:47:47] Quit
keffo__ has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [11:47:59] <sd88g93> yeah, that’s the problem
[Wednesday 08 September 2010] [12:18:38] Quit
keffo__ has left this server (Ping timeout: 264 seconds).
[Wednesday 08 September 2010] [12:33:04] <cremes> sd88g93: i recommend instrumenting the forwarder device; just have it print what it receives and what it sends out again
[Wednesday 08 September 2010] [12:33:28] <cremes> if you are using multi-part messages with the topic in the first part, then perhaps it isn’t getting forwarded correctly and
[Wednesday 08 September 2010] [12:33:43] <cremes> your subscribers are dropping the messages because the topic doesn’t match
[Wednesday 08 September 2010] [12:43:22] <erickt> are any of the pyzmq devs around today?
[Wednesday 08 September 2010] [12:44:53] Quit
keffo__ has left this server (Ping timeout: 258 seconds).
[Wednesday 08 September 2010] [13:12:48] Join ModusPwnens has joined this channel (
~Slash@75.11.168.223).
[Wednesday 08 September 2010] [13:13:08] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [13:20:07] <ModusPwnens> hmm cremes, yesterday you said that if the performance tests were working correctly, remote_thr should be the first one to exist
[Wednesday 08 September 2010] [13:22:20] <ModusPwnens> exit*
[Wednesday 08 September 2010] [13:22:36] <cremes> right; the publisher should be done before the subscriber
[Wednesday 08 September 2010] [13:22:52] <cremes> but that doesn’t always happen on my machine so i’m a tad confused
[Wednesday 08 September 2010] [13:23:09] <cremes> unfortunately, i don’t have time to look at it right now... maybe someone else can take a peek
[Wednesday 08 September 2010] [13:23:10] <ModusPwnens> Yeah, what I am seeing is the remote continuing to run
[Wednesday 08 September 2010] [13:23:15] <ModusPwnens> until it times out or something
[Wednesday 08 September 2010] [13:24:37] <cremes> i recommend you add some print statements to the tests and run them with low iterations; perhaps something obvious will pop out
[Wednesday 08 September 2010] [13:29:03] <ModusPwnens> ok, will do
[Wednesday 08 September 2010] [13:30:08] Quit gebi has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [14:20:58] Quit
keffo__ has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [14:31:08] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [14:35:55] Quit xla has left this server (Quit: leaving).
[Wednesday 08 September 2010] [14:52:27] Quit
keffo__ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [14:53:02] <mikko> hi
[Wednesday 08 September 2010] [14:55:21] <mikko> what happened with zmq_close semantics?
[Wednesday 08 September 2010] [15:02:30] Quit
keffo__ has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [15:21:20] <mato> mikko: the infrastructure work is all on ‘master’
[Wednesday 08 September 2010] [15:21:36] <mato> mikko: but i’ve not yet had time to test it properly due to “real” work :-(
[Wednesday 08 September 2010] [15:39:25] Quit
keffo__ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [16:20:18] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [16:43:27] Quit
bennymack_ has left this server (Client Quit).
[Wednesday 08 September 2010] [16:45:29] <bennymack> hey everybody! So, i’m looking into 0MQ a bit and so I was strace() a simple example in perl and I noticed a lot of these 24byte send/recv calls on the pipes between the main thread and the 0MQ context thread
[Wednesday 08 September 2010] [16:45:54] <bennymack> is there something I can read about all that chatter that’s going on?
[Wednesday 08 September 2010] [16:46:07] <bennymack> is it the SPB?
[Wednesday 08 September 2010] [17:01:53] Quit GeekGod has left this server (Ping timeout: 276 seconds).
[Wednesday 08 September 2010] [17:01:53] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 08 September 2010] [17:03:33] Quit
GeekGod_ has left this server (Changing host).
[Wednesday 08 September 2010] [17:03:33] Nick
GeekGod_ is now known as GeekGod.
[Wednesday 08 September 2010] [17:09:37] <sd88g93> alhtough i dont think it gets into the low level stuff that youre talking about
[Wednesday 08 September 2010] [17:11:35] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [17:12:45] <mato> bennymack: what you see is signalling between the application thread and zmq I/O thread, see zmq::signaler_t and read the source :-)
[Wednesday 08 September 2010] [17:12:46] <sd88g93> we were talking about it this morning, (or for you , i guess that would be afternoon)
[Wednesday 08 September 2010] [17:13:29] <sd88g93> oh, mato, you were here this morning, i managed to replicate the error i was having in this program :
http://pastebin.com/5Lsy47Cj[Wednesday 08 September 2010] [17:14:12] <mato> sd88g93: hmm, that’s long, do you have a 5-second summary?
[Wednesday 08 September 2010] [17:14:19] <sd88g93> ok,
[Wednesday 08 September 2010] [17:14:45] <sd88g93> the top program is fields requests from a client skt, and feeds them to different threads
[Wednesday 08 September 2010] [17:15:02] <sd88g93> it relays the multipart messages to a pub skt
[Wednesday 08 September 2010] [17:15:19] <sd88g93> (the second program is a client app to send it messages )
[Wednesday 08 September 2010] [17:15:52] <sd88g93> the problem is in the top program, it recives the messages fine , but doesnt relay the messages to the central publisher socket in the main thread
[Wednesday 08 September 2010] [17:16:33] <sd88g93> they should be easy to compile, just gcc -otestpub testpub.c -lzmq for the top program
[Wednesday 08 September 2010] [17:16:36] <mato> sd88g93: so did you find your problem?
[Wednesday 08 September 2010] [17:16:56] <sd88g93> i replicated the same problem in that code
[Wednesday 08 September 2010] [17:17:16] <sd88g93> its in the loop in the top program, (in the thread)
[Wednesday 08 September 2010] [17:17:19] <mato> sorry, it’s kind of late here ...
[Wednesday 08 September 2010] [17:17:25] <sd88g93> oh ok no problem
[Wednesday 08 September 2010] [17:17:26] <bennymack> ok thanks! I’ll take a look
[Wednesday 08 September 2010] [17:17:43] <sd88g93> I can post to the mailing list
[Wednesday 08 September 2010] [17:17:52] <mato> that might be best
[Wednesday 08 September 2010] [17:18:04] <sd88g93> yeah, still 5p.m. here
[Wednesday 08 September 2010] [17:18:09] <jond> sd88g93: is this the same problem as the code on the mail list. that needed a sleep after the pthread_create?
[Wednesday 08 September 2010] [17:18:33] <sd88g93> it might be, not sure which one youre thinking of
[Wednesday 08 September 2010] [17:18:40] <mato> jond: hi Jon... am in the middle of replying to your mail re Git
[Wednesday 08 September 2010] [17:19:16] <mato> jond: is your patmatch branch a private topic branch? i.e. noone pulling from it?
[Wednesday 08 September 2010] [17:19:41] <jond> mato: hi , thanks, also what does ‘Your branch is behind ‘origin/master’ by 86 commits, and can be fast-forwarded.’ mean?
[Wednesday 08 September 2010] [17:20:34] <jond> mato: yes, patmatch private topic branch which I intend to run format patch from and all be hunky dory
[Wednesday 08 September 2010] [17:20:36] <mato> jond: if it’s your local ‘master’ branch that means you can pull from it and you’re 86 commits behind...
[Wednesday 08 September 2010] [17:21:06] <mato> jond: the “can fast forward” stuff basically means you’ve not made any commits to your local master branch that would require a merge
[Wednesday 08 September 2010] [17:21:34] <jond> mato: yep local master; It;s been behind every since I did a hard reset that chuck posted, but i’m not clear how to get rid of it
[Wednesday 08 September 2010] [17:21:54] <mato> jond: git pull
[Wednesday 08 September 2010] [17:22:12] <jond> mato: this is after a git pull
[Wednesday 08 September 2010] [17:22:16] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Wednesday 08 September 2010] [17:23:00] <jond> mato: git pull says this now error: Untracked working tree file ‘src/connect_session.cpp’ would be overwritten by merge.
[Wednesday 08 September 2010] [17:23:22] <mato> jond: oh, right, you have changes you’ve not committed on local master
[Wednesday 08 September 2010] [17:23:28] <mato> jond: what do you want to do with those?
[Wednesday 08 September 2010] [17:24:29] <mato> jond: git status will show you what git thinks about your working tree state
[Wednesday 08 September 2010] [17:24:36] <jond> mato: there shouldnt be any, this all started when I pulled the 2.1 tree then reset to 2.0.8. I havent created those files or edited?
[Wednesday 08 September 2010] [17:24:55] <mato> jond: what does git status say?
[Wednesday 08 September 2010] [17:25:33] <mato> jond: alternatively, do you have any changes you care about in that repository?
[Wednesday 08 September 2010] [17:25:33] <jond> mato: git status gives me the 86 commits behind and some untracked files ; do i need to remove those?
[Wednesday 08 September 2010] [17:27:00] <jond> mato: thats fixed master
[Wednesday 08 September 2010] [17:27:10] <mato> ?
[Wednesday 08 September 2010] [17:27:33] <jond> i removed the ‘overwritten’ files
[Wednesday 08 September 2010] [17:27:48] <mato> you probably wanted git checkout – <path>
[Wednesday 08 September 2010] [17:28:03] <mato> does git status now say “working directory clean” ?
[Wednesday 08 September 2010] [17:29:34] <jond> mato: yes, after I removed some other note files I had left there
[Wednesday 08 September 2010] [17:29:55] <mato> jond: goodo, then you should be able to pull, and follow the instructions in my email
[Wednesday 08 September 2010] [17:30:10] <mato> jond: but if you want to be sure you don’t mess up just take a backup copy of your local repo
[Wednesday 08 September 2010] [17:30:20] <mato> jond: (cp -r zeromq2 zeromq2.bak or something)
[Wednesday 08 September 2010] [17:30:30] <jond> mato: will do , many thanks
[Wednesday 08 September 2010] [17:30:53] <mato> ok, i’m off... cyl
[Wednesday 08 September 2010] [17:31:01] <jond> mato: cyl
[Wednesday 08 September 2010] [17:31:09] <sd88g93> good night , mato
[Wednesday 08 September 2010] [17:31:15] <mato> nite, thx
[Wednesday 08 September 2010] [17:31:16] <sd88g93> ok, just posted to the ML
[Wednesday 08 September 2010] [17:36:42] Part sd88g93 has left this channel (“www.phash.org”).
[Wednesday 08 September 2010] [17:39:00] Quit andrewvc has left this server (Quit: andrewvc).
*** on Wed Sep 8 20:16:49 2010
[Wednesday 08 September 2010] [20:16:49] Topic The channel topic is “Welcome!”.
[Wednesday 08 September 2010] [20:16:55] Mode Channel modes: no messages from outside, topic protection
[Wednesday 08 September 2010] [20:16:55] Created This channel was created on 2010-02-05 10:44.
*** on Thu Sep 9 14:21:58 2010
[Thursday 09 September 2010] [14:21:58] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [14:22:04] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [14:22:04] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [14:30:21] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [14:30:23] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [14:30:24] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [14:30:24] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [14:30:31] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [14:30:31] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [14:30:48] <pieterh> PerfDave, sorry, i was kidding around
[Thursday 09 September 2010] [14:30:53] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [14:31:42] <pieterh> sd88g93, ok, test case is now clear, lemme think about it a sec
*** on Thu Sep 9 15:44:58 2010
[Thursday 09 September 2010] [15:44:58] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [15:45:04] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [15:45:04] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [15:45:13] <pieterh> sd88g93, what is ‘qskt’ for?
[Thursday 09 September 2010] [15:45:31] <sd88g93> that’s the rep socket, in the worker thread, that recieves the messages
[Thursday 09 September 2010] [15:45:35] <sd88g93> from the client
[Thursday 09 September 2010] [15:45:36] <pieterh> there is already a terrible lack of vowels here in Slovakia
[Thursday 09 September 2010] [15:46:06] <sd88g93> tough times, gotta conservee
[Thursday 09 September 2010] [15:46:22] <pieterh> that’s kind of like “who is that guy?” answer: “two legs, body, arms, head”
[Thursday 09 September 2010] [15:47:02] <pieterh> cause it does not in fact talk to the client at all
[Thursday 09 September 2010] [15:47:02] <sd88g93> couldnt come up with a more descriptive variable, sorry, i blame my lack of imagination
[Thursday 09 September 2010] [15:47:19] <sd88g93> it doesnt ? it does on mine
[Thursday 09 September 2010] [15:47:25] <pieterh> it talks to the main thread
[Thursday 09 September 2010] [15:47:26] <pieterh> via inproc
[Thursday 09 September 2010] [15:47:45] <sd88g93> yeah, then there’s a queue device to the main socket
[Thursday 09 September 2010] [15:47:47] <sd88g93> the client skt
[Thursday 09 September 2010] [15:48:11] <pieterh> why is the client not talking to the worker directly?
[Thursday 09 September 2010] [15:48:28] <sd88g93> if you have more than one thread
[Thursday 09 September 2010] [15:48:44] <sd88g93> use the device , at least that’s the scenario
[Thursday 09 September 2010] [15:48:55] <pieterh> well...
[Thursday 09 September 2010] [15:49:00] <pieterh> it
[Thursday 09 September 2010] [15:49:15] <pieterh> the device is really when you have an N-to-N case
[Thursday 09 September 2010] [15:49:23] <pieterh> and don’t want all the N’s to have to know about each other
[Thursday 09 September 2010] [15:49:43] <sd88g93> yeah, there’s just one client now, but there could be more
[Thursday 09 September 2010] [15:49:46] <pieterh> ah... you are in fact developing the MT server model...
[Thursday 09 September 2010] [15:49:54] <sd88g93> yeah
[Thursday 09 September 2010] [15:49:57] <pieterh> i hardly recognized it... :-/
[Thursday 09 September 2010] [15:49:59] <sd88g93> there could be more than one client
[Thursday 09 September 2010] [15:50:58] <sd88g93> well, a diffferent form of it, i guess
[Thursday 09 September 2010] [15:53:00] <sd88g93> zhelpers.h:25 i thought sprintf was “evil”, suppose to use snprintf ?
[Thursday 09 September 2010] [15:54:16] <keffo> pieterh, I’ve done the lru now!
[Thursday 09 September 2010] [15:54:34] <pieterh> keffo: wow!
[Thursday 09 September 2010] [15:54:41] <pieterh> sd88g93, this is what the worker loop becomes:
[Thursday 09 September 2010] [15:54:44] <pieterh> char
*command = s_recv (control);
[Thursday 09 September 2010] [15:54:44] <pieterh> char
*count = s_recv (control);
[Thursday 09 September 2010] [15:54:44] <pieterh> puts (“Worker got command: %sn”, command);
[Thursday 09 September 2010] [15:54:44] <pieterh> s_send (publisher, “msg.x|”);
[Thursday 09 September 2010] [15:54:47] <pieterh> s_sendmore (publisher, command);
[Thursday 09 September 2010] [15:54:47] <pieterh> s_send (publisher, count);
[Thursday 09 September 2010] [15:54:48] <pieterh> free (command);
[Thursday 09 September 2010] [15:54:50] <pieterh> free (count);
[Thursday 09 September 2010] [15:54:52] <pieterh> s_send (control, “OK”);
[Thursday 09 September 2010] [15:54:59] <pieterh> there is a bug, i see immediately...
[Thursday 09 September 2010] [15:55:03] <pieterh> in my code, not yours
[Thursday 09 September 2010] [15:55:24] <ModusPwnens> Wait a sec pieter, i was referring to something on the site
[Thursday 09 September 2010] [15:55:49] <sd88g93> oh, in s_send and s_sendmore you do init the message and close it each time,
[Thursday 09 September 2010] [15:56:16] <sd88g93> yeah, that’s what i thought, you were just referrring to coding style, eh ? not actuall instructions needed
[Thursday 09 September 2010] [15:56:24] <sd88g93> i get ya now, ok
[Thursday 09 September 2010] [15:56:50] <pieterh> ModusPwnens, even better, join the site and click ‘edit’
[Thursday 09 September 2010] [15:56:57] <pieterh> it’s a wiki :-)
[Thursday 09 September 2010] [15:57:07] <ModusPwnens> Oh okay, well that’s easy to fix then
[Thursday 09 September 2010] [15:57:09] <keffo> pieterh, it seems to be working very well.. I just need to post back the results now, but I’ll probably use a new socketlayer for that, with some persistance as well
[Thursday 09 September 2010] [15:57:25] <pieterh> hmm, nice
[Thursday 09 September 2010] [15:57:38] <pieterh> keffo: i still find working with envelopes to be too painful
[Thursday 09 September 2010] [15:57:52] <pieterh> but we’ll make abstractions for those later
[Thursday 09 September 2010] [15:58:47] <keffo> I have a Route class for that
[Thursday 09 September 2010] [15:59:02] <pieterh> yeah, right, you told me... nice
[Thursday 09 September 2010] [15:59:04] <keffo> which can construct itself from incoming, and also send it
[Thursday 09 September 2010] [15:59:07] <keffo> yeah
[Thursday 09 September 2010] [15:59:19] <pieterh> were the names in the routing section too weird?
[Thursday 09 September 2010] [15:59:27] <pieterh> i was trying to get mnemonics that worked
[Thursday 09 September 2010] [15:59:27] <keffo> where?
[Thursday 09 September 2010] [15:59:31] <pieterh> ch3
[Thursday 09 September 2010] [15:59:32] <keffo> mama,papa? =)
[Thursday 09 September 2010] [15:59:41] <pieterh> yeah
[Thursday 09 September 2010] [15:59:49] <jhawk28> the mama and papa were hilarious
[Thursday 09 September 2010] [15:59:52] <keffo> papa was a bit confusing, didnt really see the application to be honest, but I was mostly interested in the mama anyway :)
[Thursday 09 September 2010] [16:00:14] <pieterh> well, good, it’s hard to make this material even remotely funny
[Thursday 09 September 2010] [16:00:25] * pieterh still has to write a short piece on quick and clean exits
[Thursday 09 September 2010] [16:00:36] <jhawk28> we enjoyed it at work
[Thursday 09 September 2010] [16:00:37] <psino> pieterh: you should move some more reasoning about the naming in ch3 to the place where you introduce them
[Thursday 09 September 2010] [16:00:46] <keffo> pieterh, Another interesting sideffect of the mama topology you presented there is that there is no longer any immediate necessity for a heartbeat system
[Thursday 09 September 2010] [16:00:53] <pieterh> psino: did I get the order wrong?
[Thursday 09 September 2010] [16:01:07] <pieterh> yup, but you need heartbeats for another reason
[Thursday 09 September 2010] [16:01:26] <pieterh> i was going to write that today but got distracted by our friend sd88g93’s curious case here
[Thursday 09 September 2010] [16:01:36] <pieterh> also, ISP guys getting lost
[Thursday 09 September 2010] [16:01:38] <keffo> sure, but not in the same way as a topdown design
[Thursday 09 September 2010] [16:02:03] <psino> no, but when you say that REQ is a MAMA socket, making insistent demands, you might want to add something more that helps the reader remember the terminology
[Thursday 09 September 2010] [16:02:26] <pieterh> hmm, lemme think about that for a second...
[Thursday 09 September 2010] [16:02:52] <pieterh> i wrote that text very late on tuesday evening after a glass or two of wine... :-/
[Thursday 09 September 2010] [16:03:05] <psino> you describe the usage of mama pretty good in the “Router-to-Mama”-section
[Thursday 09 September 2010] [16:03:23] <psino> (personal opinion of course)
[Thursday 09 September 2010] [16:03:30] <pieterh> i think the character is based on my friend Fabio’ Italian mother
[Thursday 09 September 2010] [16:03:36] <pieterh> ok...
[Thursday 09 September 2010] [16:03:44] <pieterh> opinion is better than silence
[Thursday 09 September 2010] [16:04:33] <jhawk28> I was going through the swap and hwm parts of the user guide and I got an exception after the pub ran for a while (from yquery:108)
[Thursday 09 September 2010] [16:05:13] <pieterh> jhawk28, interesting!
[Thursday 09 September 2010] [16:05:23] <jhawk28> It died when I was using the connect, but not the bind
[Thursday 09 September 2010] [16:06:03] <jhawk28> not sure if it matters, but I was using the Java bindings
[Thursday 09 September 2010] [16:06:14] <pieterh> do you mean yqueue?
[Thursday 09 September 2010] [16:06:30] <jhawk28> yes, it was an assertion that failed
[Thursday 09 September 2010] [16:06:52] <jhawk28> going from memory - so it could be wrong
[Thursday 09 September 2010] [16:06:58] <pieterh> could you please log this on the issue tracker, along with the hardware setup you’re using?
[Thursday 09 September 2010] [16:07:12] <jhawk28> I think so
[Thursday 09 September 2010] [16:07:12] <pieterh> this should absolutely never never happen
[Thursday 09 September 2010] [16:07:27] <jhawk28> figured as much
[Thursday 09 September 2010] [16:07:30] <pieterh> there is indeed an assertion on yqueue.hpp:108
[Thursday 09 September 2010] [16:07:42] <pieterh> woa, hang on
[Thursday 09 September 2010] [16:07:49] <pieterh> it’s an out of memory condition
[Thursday 09 September 2010] [16:08:36] <jhawk28> hwm was set to 1 and swap was set to 25000000
[Thursday 09 September 2010] [16:09:27] <pieterh> you sure you didn’t reverse those? :-)
[Thursday 09 September 2010] [16:09:54] <pieterh> it’s worth logging, we’ll run that on some big boxes here and see if we can reproduce it
[Thursday 09 September 2010] [16:09:56] <jhawk28> :) was following the guide and then started playing around seeing what it would do
[Thursday 09 September 2010] [16:09:59] <pieterh> thanks
[Thursday 09 September 2010] [16:14:28] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [16:14:29] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [16:14:30] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [16:14:31] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [16:14:37] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [16:14:37] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [16:14:48] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [16:15:06] <daggertip> so
[Thursday 09 September 2010] [16:16:15] Part daggertip has left this channel.
[Thursday 09 September 2010] [16:17:16] * pieterh salutes the short-lived but nicely named daggertip
[Thursday 09 September 2010] [16:17:36] * cremes ha
[Thursday 09 September 2010] [16:23:10] <pieterh> sd88g93, you still there? I award you the “obfuscated 0MQ contest” award :-)
[Thursday 09 September 2010] [16:24:33] <pieterh> you have six sockets and two devices in your minimal test case
[Thursday 09 September 2010] [16:45:19] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [16:45:20] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [16:45:21] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [16:45:27] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [16:45:27] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [16:46:41] <sd88g93> i’m always up for an award
[Thursday 09 September 2010] [16:47:41] <sd88g93> think sustrik can have a looksie ?
[Thursday 09 September 2010] [16:54:15] <pieterh> sd88g93, lol, are you trolling me?
[Thursday 09 September 2010] [16:54:34] <sd88g93> trolling ?
[Thursday 09 September 2010] [16:54:36] <pieterh> i’ve stripped it right down and removing the queue device lets it work
[Thursday 09 September 2010] [16:54:39] <pieterh> :-)
[Thursday 09 September 2010] [16:54:56] <pieterh> gnash, i will find this... gnash
[Thursday 09 September 2010] [16:55:14] <sd88g93> how do you read the messages into the thread ?
[Thursday 09 September 2010] [16:55:28] <pieterh> well, connect worker directly to client
[Thursday 09 September 2010] [16:55:47] <pieterh> next time, develop such code step by step so you can test it gradually
[Thursday 09 September 2010] [16:56:03] <pieterh> and when you add a new piece if it does not work you know exactly what is questionable
[Thursday 09 September 2010] [16:56:07] <sd88g93> i’m suspicious that the messages are being altered in some way, like martin is slipping coded messages in there
[Thursday 09 September 2010] [16:56:18] <sd88g93> yeah, that’s what i did , but ....
[Thursday 09 September 2010] [16:56:24] <pieterh> i doubt it
[Thursday 09 September 2010] [16:56:29] <sd88g93> I realized zmq can do more
[Thursday 09 September 2010] [16:57:10] <sd88g93> I had a little different setup before, and then i put the forwarder back in , because i really didnt want to move zmq_msg_t ‘s around between threads
[Thursday 09 September 2010] [16:57:24] <pieterh> that is a messed up version but it shows you what it looks like
[Thursday 09 September 2010] [16:57:47] <pieterh> see how the queue thread creates its own sockets
[Thursday 09 September 2010] [16:58:06] <pieterh> not necessary to do that in main, kind of pointless and... in fact... illegal
[Thursday 09 September 2010] [16:58:15] <pieterh> you cannot create socket in thread A and use it in thread B
[Thursday 09 September 2010] [17:01:47] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [17:01:49] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [17:01:50] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:01:50] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:01:57] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:01:57] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:02:10] Quit
travlr_ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:05:13] <pieterh> ok, sd88g93 that’s the problem
[Thursday 09 September 2010] [17:05:30] <sd88g93> oh really
[Thursday 09 September 2010] [17:06:10] <pieterh> dude, you owe me a case of beer
[Thursday 09 September 2010] [17:06:36] <sd88g93> yes yes, i do hahaa
[Thursday 09 September 2010] [17:06:48] <pieterh> the publisher socket is not there, you can put it back of course
[Thursday 09 September 2010] [17:07:07] <pieterh> so problem was that your queue thread wasn’t working reliably
[Thursday 09 September 2010] [17:07:21] <pieterh> when you test code, don’t run it once, run it 10-100 times
[Thursday 09 September 2010] [17:07:31] <sd88g93> so you just create the skts for the device in that thread then,
[Thursday 09 September 2010] [17:07:35] <pieterh> threading issues are nasty like that
[Thursday 09 September 2010] [17:07:47] <pieterh> pls dnt sy skts t mks m brn hrt
[Thursday 09 September 2010] [17:08:01] <pieterh> socks if you really have to save on letters
[Thursday 09 September 2010] [17:08:12] <pieterh> but yes, you do
[Thursday 09 September 2010] [17:08:15] <sd88g93> what’s wrong with skts ?
[Thursday 09 September 2010] [17:08:32] <pieterh> what’s wrong with the letters ‘o’ and ‘e’?
[Thursday 09 September 2010] [17:08:33] <pieterh> :-)
[Thursday 09 September 2010] [17:08:52] <pieterh> seriously, if you remove essential letters from words you create extra parsing costs for the reader
[Thursday 09 September 2010] [17:08:59] <sd88g93> sorry, that’s just how i do variables, i tend to leave out vowels
[Thursday 09 September 2010] [17:09:08] <pieterh> english may look redundant but it’s been elegantly evolved for lazy readers
[Thursday 09 September 2010] [17:09:25] <pieterh> if you want a hint for style: make your code readable
[Thursday 09 September 2010] [17:09:37] <pieterh> please compare my versions with yours
[Thursday 09 September 2010] [17:09:58] <sd88g93> those zhelper.h are helpful, but unfortunately, i need it for binary data
[Thursday 09 September 2010] [17:10:01] <pieterh> you seriously cost me several hours just to untangle your code...
[Thursday 09 September 2010] [17:10:16] <pieterh> zhelpers.h is an example, modify/extend it for binary data
[Thursday 09 September 2010] [17:10:20] Quit psino has left this server (Quit: psino).
[Thursday 09 September 2010] [17:10:23] <pieterh> s_recv already works for binary data
[Thursday 09 September 2010] [17:10:30] <sd88g93> well, i was hoping for zero copy as well
[Thursday 09 September 2010] [17:10:32] <pieterh> add s_send_bin (socket, data, size) and it’s done
[Thursday 09 September 2010] [17:10:48] <sd88g93> ok,
[Thursday 09 September 2010] [17:10:58] <pieterh> just make sure to copy the data
[Thursday 09 September 2010] [17:11:04] <pieterh> otherwise you’ll get weirdness too
[Thursday 09 September 2010] [17:11:44] <pieterh> unless you are doing a lot of traffic (100K+ / second) or very large messages, zerocopy won’t be an advantage
[Thursday 09 September 2010] [17:11:52] <pieterh> but it will make your code harder to write and read and fix
[Thursday 09 September 2010] [17:11:53] <sd88g93> well, there’s no way to do zero copy though, when i can find a way to do zerocopy and those functions , then we’ve got something
[Thursday 09 September 2010] [17:12:09] <pieterh> do not optimize code that isn’t working
[Thursday 09 September 2010] [17:12:17] <pieterh> it is really a bad idea
[Thursday 09 September 2010] [17:12:31] <pieterh> actually I need to totally remove all reference to zerocopy from the user guide
[Thursday 09 September 2010] [17:12:37] <sd88g93> well, one message part can be pretty big
[Thursday 09 September 2010] [17:12:41] <pieterh> people should not even know it exists until chapter 4 or so...
[Thursday 09 September 2010] [17:12:47] <pieterh> doesn’t matter
[Thursday 09 September 2010] [17:12:55] <sd88g93> like 100k
[Thursday 09 September 2010] [17:13:02] <pieterh> doesn’t matter
[Thursday 09 September 2010] [17:13:03] <sd88g93> 100KB , easily
[Thursday 09 September 2010] [17:13:07] <pieterh> seriously, do the test yourself
[Thursday 09 September 2010] [17:13:21] <sd88g93> really ? it wont matter ?
[Thursday 09 September 2010] [17:13:36] <pieterh> for (namewithvowels = 0; nwv < 1000; nwv++) { do large copy here }
[Thursday 09 September 2010] [17:13:39] <sd88g93> hmm ok , i’ll have to think on that some
[Thursday 09 September 2010] [17:13:41] <pieterh> run, test, prove it to yourself
[Thursday 09 September 2010] [17:13:58] <pieterh> it matters if you are into microsecond latency issues and CPUs are running at 75% full
[Thursday 09 September 2010] [17:14:11] <pieterh> i’ll bet you that case of beer x 2 that’s not your case
[Thursday 09 September 2010] [17:14:41] <pieterh> premature optimization is one of the many roads to evil and it’s not even a particularly fun one
[Thursday 09 September 2010] [17:15:07] <gebi> zero-copy can also easily be more expensive than plain copy if not aligned on cache-lines and between different cpus
[Thursday 09 September 2010] [17:18:26] <pieterh> ok, folk, /me has to go sleepy byes
[Thursday 09 September 2010] [17:22:16] Nick Nickname already in use. Trying travlr.
[Thursday 09 September 2010] [17:22:17] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Thursday 09 September 2010] [17:22:17] Notice -NickServ- You are now identified for travlr.
[Thursday 09 September 2010] [17:22:18] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:22:19] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:22:25] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:22:25] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:22:57] Quit
travlr__ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:23:26] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:24:28] Quit sd88g93 has left this server (Quit: Ex-Chat).
[Thursday 09 September 2010] [17:31:42] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [17:31:44] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [17:31:44] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:31:45] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:31:48] Quit travlr has left this server (Ping timeout: 245 seconds).
[Thursday 09 September 2010] [17:31:52] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:31:52] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:55:12] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [17:55:14] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [17:55:15] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:55:15] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:55:22] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:55:22] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:56:18] Quit
travlr_ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [18:40:07] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [18:40:08] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [18:40:09] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [18:40:15] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [18:40:15] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [18:42:04] Quit kelvink has left this server (Quit: Leaving.).
[Thursday 09 September 2010] [18:59:09] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:04:37] Nick Nickname already in use. Trying travlr.
[Thursday 09 September 2010] [19:04:39] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Thursday 09 September 2010] [19:04:39] Notice -NickServ- You are now identified for travlr.
[Thursday 09 September 2010] [19:04:39] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:04:40] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:04:47] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:04:47] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:04:56] Quit
travlr__ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:15:04] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [19:15:06] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [19:15:06] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:15:07] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:15:14] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:15:14] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:15:34] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:25:31] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [19:25:33] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [19:25:33] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:25:34] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:25:40] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:25:41] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:26:23] Quit
travlr_ has left this server (Ping timeout: 276 seconds).
*** on Thu Sep 9 20:09:53 2010
[Thursday 09 September 2010] [20:09:53] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [20:09:59] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [20:09:59] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [20:18:18] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [20:18:24] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [20:18:24] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [20:19:11] Quit travlr has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 11 03:24:23 2010
[Saturday 11 September 2010] [03:24:23] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [03:24:29] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [03:24:29] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [03:31:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [03:53:07] Quit ivan has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [04:08:05] Quit rgl has left this server (Ping timeout: 272 seconds).
[Saturday 11 September 2010] [04:16:31] Join gebi has joined this channel (
~gebi@84.119.81.11).
[Saturday 11 September 2010] [05:22:59] Quit ivan has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 11 06:07:38 2010
[Saturday 11 September 2010] [06:07:38] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [06:07:44] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [06:07:44] Created This channel was created on 2010-02-05 10:44.
*** on Sat Sep 11 06:32:47 2010
[Saturday 11 September 2010] [06:32:47] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [06:32:53] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [06:32:53] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [07:44:15] <icy> I can connect a socket to many endpoints, what is the reason for not allowing one to disconnect from an endpoint again?
[Saturday 11 September 2010] [08:00:29] <icy> also why do I have to loop over all sockets to see which ones got events after a zmq_poll(), I can’t see how this would scale
[Saturday 11 September 2010] [08:03:36] <guido_g> that’s how poll works
[Saturday 11 September 2010] [08:08:57] <icy> well that kinda nullifies a benefit that for example epoll() gives you
[Saturday 11 September 2010] [08:09:05] <icy> where you don’t have to loop around searching
[Saturday 11 September 2010] [08:11:16] <guido_g> a) somewhere you have to go through the list of fds
[Saturday 11 September 2010] [08:12:11] <guido_g> b) mq sockets aggregate os level sockets (one mq socket -> many os level sockets), so you have only a few mq sockets
[Saturday 11 September 2010] [08:12:37] <guido_g> c) mq uses epoll or whatever is available on the given platform
[Saturday 11 September 2010] [08:12:42] <icy> yea but due to not being able to disconnect from one endpoint, I might end up needing many zmq sockets
[Saturday 11 September 2010] [08:13:07] <guido_g> i don’t dare to ask why...
[Saturday 11 September 2010] [08:13:08] <icy> it uses epoll but makes me loop over all sockets, epoll does not require me to do that
[Saturday 11 September 2010] [08:13:45] <guido_g> epoll is not available on all platforms mq is running on
[Saturday 11 September 2010] [08:13:49] <icy> I know
[Saturday 11 September 2010] [08:13:52] <guido_g> so?
[Saturday 11 September 2010] [08:14:02] <icy> but many provide smarter facilities than poll()
[Saturday 11 September 2010] [08:14:44] <icy> so you throw away a lot of benefits for a lowest common denominator
[Saturday 11 September 2010] [08:15:03] <guido_g> so you think it would make sense to have mq behave differently on differnt platforms?
[Saturday 11 September 2010] [08:15:17] <icy> that is not what I am saying
[Saturday 11 September 2010] [08:15:21] <guido_g> it is
[Saturday 11 September 2010] [08:15:27] <icy> nope
[Saturday 11 September 2010] [08:15:55] <guido_g> because if you use the _best_ way on a platform, you also have the platform specific semantics
[Saturday 11 September 2010] [08:16:15] <icy> you can still abstract it in a general way
[Saturday 11 September 2010] [08:16:24] <icy> see all event libraries like libev or libevent
[Saturday 11 September 2010] [08:16:56] <gebi> icy: just for the record, epoll is NOT faster than poll in some situations, quite frankly it is 20% or more slower than poll in some situations
[Saturday 11 September 2010] [08:17:11] <icy> I know, for small sets poll() is nice
[Saturday 11 September 2010] [08:17:35] <guido_g> may be these abstraction would cost a lot on some platforms, so why punish people not running you favorite platform?
[Saturday 11 September 2010] [08:18:05] <icy> they do not cost any considerable amount
[Saturday 11 September 2010] [08:18:46] <guido_g> evidence by allegation?
[Saturday 11 September 2010] [08:19:01] <icy> what evidence do you provide?
[Saturday 11 September 2010] [08:19:48] <guido_g> mq is built that way, and works
[Saturday 11 September 2010] [08:21:32] <icy> well all I’m saying is I cannot see how it can scale to many zmq sockets because of all the looping. so clearly there must be a reason for that and that would that you would not need many zqm sockets as each can connect to many transport endpoints
[Saturday 11 September 2010] [08:22:07] <icy> which brings me to my initial question why you can’t disconnect from one again
[Saturday 11 September 2010] [08:23:15] <icy> I might be missing something in how you are supposed to handle many volatile connections and am ready for enlightenment :)
[Saturday 11 September 2010] [08:25:11] <guido_g> what kind of connections?
[Saturday 11 September 2010] [08:25:32] <guido_g> you don’t see connects at the mq level
[Saturday 11 September 2010] [08:28:26] <icy> hm I was thinking of a socket which connects to many IPs over tcp but I guess I just need to flip it around and make it the server and not client
[Saturday 11 September 2010] [08:29:25] <guido_g> even then you would have only one mq socket
[Saturday 11 September 2010] [08:29:33] <pieter_hintjens> gebi: yes, i recall Zed Shaw’s analysis, quite fun
[Saturday 11 September 2010] [08:30:03] <pieter_hintjens> icy: you’re IMO trying to apply existing TCP patterns to 0MQ
[Saturday 11 September 2010] [08:30:03] <icy> yea but if the IPs are dynamic, the zmq socket would just get more and more endpoints attached to it
[Saturday 11 September 2010] [08:30:17] <icy> pieter_hintjens: quite possible, I am used to that :)
[Saturday 11 September 2010] [08:30:55] <guido_g> if you connect more and more sockets to a remote socket, then yes, the number of connections is growing
[Saturday 11 September 2010] [08:30:57] <pieter_hintjens> icy: you need to think of 0MQ sockets as mini applications not just APIs
[Saturday 11 September 2010] [08:31:08] <pieter_hintjens> they are intelligent and do things for you
[Saturday 11 September 2010] [08:31:24] <pieter_hintjens> learning this is the key to using 0MQ right and getting it to scale and be useful for you
[Saturday 11 September 2010] [08:31:54] <pieter_hintjens> there are some generic rules that apply to all 0MQ sockets
[Saturday 11 September 2010] [08:32:02] <pieter_hintjens> and then there are specifics to each type, how they behave
[Saturday 11 September 2010] [08:32:12] <pieter_hintjens> generically, client vs. server is important to get right
[Saturday 11 September 2010] [08:32:35] <pieter_hintjens> stable nodes act better as servers (doing the bind), dynamic nodes as clients (doing the connect)
[Saturday 11 September 2010] [08:32:36] <icy> yup makes sense, I think it will just take a bit of time to get the old patterns out of my head
[Saturday 11 September 2010] [08:32:39] <pieter_hintjens> yup
[Saturday 11 September 2010] [08:32:50] <pieter_hintjens> if it’s any help, we all go through this
[Saturday 11 September 2010] [08:32:59] <pieter_hintjens> did you read the user guide and try the examples?
[Saturday 11 September 2010] [08:33:05] <icy> yes
[Saturday 11 September 2010] [08:33:16] <pieter_hintjens> did you just read it, or try the code?
[Saturday 11 September 2010] [08:33:36] <icy> I tried not all but some
[Saturday 11 September 2010] [08:33:38] <icy> and read all
[Saturday 11 September 2010] [08:33:44] <pieter_hintjens> well, it takes time too
[Saturday 11 September 2010] [08:33:58] <pieter_hintjens> several days of slow reprogramming
[Saturday 11 September 2010] [08:34:02] <guido_g> it needs time to sink in
[Saturday 11 September 2010] [08:34:15] <guido_g> mq can’t be groked in a day, imho
[Saturday 11 September 2010] [08:34:46] <pieter_hintjens> if you never did networking programming you’d have less trouble unlearning old patterns
[Saturday 11 September 2010] [08:34:51] <guido_g> the things that makes it so good is what makes it so hard :)
[Saturday 11 September 2010] [08:34:53] <pieter_hintjens> but you’d still have to learn the new stuff
[Saturday 11 September 2010] [08:35:09] <guido_g> but who would that be?
[Saturday 11 September 2010] [08:35:11] <pieter_hintjens> icy: briefly, you trust 0MQ sockets to do a lot for you
[Saturday 11 September 2010] [08:35:34] <pieter_hintjens> there is a lot less of the command-and-control you need with traditional sockets
[Saturday 11 September 2010] [08:36:32] <icy> well I understand how stuff works, not very complicated after all. I’m trying now to understand why the api is how it is. e.g. normally event libs let you specify callbacks that get called upon events (hence I was so surprised by the looping needed after zmq_poll)
[Saturday 11 September 2010] [08:36:51] <pieter_hintjens> icy: the api aims to look like plain boring sockets
[Saturday 11 September 2010] [08:37:05] <pieter_hintjens> the poll stuff is not very nice
[Saturday 11 September 2010] [08:37:13] <pieter_hintjens> and we’ll probably make it nicer for 3.0
[Saturday 11 September 2010] [08:37:19] <icy> fair enough
[Saturday 11 September 2010] [08:37:37] <pieter_hintjens> there is a moral when you make new technology
[Saturday 11 September 2010] [08:37:43] <pieter_hintjens> if it looks new, people won’t adopt it
[Saturday 11 September 2010] [08:38:16] <guido_g> hmmm
[Saturday 11 September 2010] [08:38:42] <pieter_hintjens> people = except pioneers who have a fetish for new technology just on principle
[Saturday 11 September 2010] [08:38:54] <pieter_hintjens> and who will abandon it for the next new thing without hesitation
[Saturday 11 September 2010] [08:39:26] <guido_g> i hmmed because the last 30 years have seen a lot of new things
[Saturday 11 September 2010] [08:39:40] <pieter_hintjens> I read a lovely quote from the inventor of liquid salt nuclear reactors
[Saturday 11 September 2010] [08:39:54] <pieter_hintjens> sure... i’m making a horrid generalization
[Saturday 11 September 2010] [08:39:59] <guido_g> ack
[Saturday 11 September 2010] [08:39:59] <icy> I’m not so sure if this can be applied so easily
[Saturday 11 September 2010] [08:40:02] <guido_g> go on :)
[Saturday 11 September 2010] [08:40:11] <pieter_hintjens> “Perhaps the moral to be drawn is that a technology that differs too much from an existing technology has not one hurdle to overcome–to demostrate its feasibility–but another even greater one–to convince influential individuals and organizations who are intellectually and emotionally attached to a different technology that they should adopt the new path”
[Saturday 11 September 2010] [08:40:29] <guido_g> icy: it’s a trade-off between usability and speed, mostly
[Saturday 11 September 2010] [08:40:30] <icy> if it was true, no radically things would have been adopted in the past but there have been plenty :)
[Saturday 11 September 2010] [08:40:50] <pieter_hintjens> ah, (a) there are always exceptions to a general rule and (b) there are ways to appear old yet introduce newness
[Saturday 11 September 2010] [08:40:56] <guido_g> i’ve seen a messaging lib w/ callbacks, and it wasn’t fun at all
[Saturday 11 September 2010] [08:41:20] <icy> in my opinion anything providing a considerable advantage over something else has good chances of getting adopted
[Saturday 11 September 2010] [08:41:36] <icy> provided it does not have disadvantages that outweight the advantages of course :P
[Saturday 11 September 2010] [08:41:44] <guido_g> you remember the video-tape thingy?
[Saturday 11 September 2010] [08:41:46] <pieter_hintjens> this is the engineer point of view but it ignores people emotional attachment to stuff they know
[Saturday 11 September 2010] [08:41:52] <pieter_hintjens>
*people’s
[Saturday 11 September 2010] [08:41:57] <pieter_hintjens>
*peoples’
[Saturday 11 September 2010] [08:42:06] <pieter_hintjens> sms text messages
[Saturday 11 September 2010] [08:42:10] <pieter_hintjens> email
[Saturday 11 September 2010] [08:42:10] <icy> guido_g: what about fire, electricity etc? :)
[Saturday 11 September 2010] [08:42:22] <pieter_hintjens> icy: fire has callbacks?
[Saturday 11 September 2010] [08:42:30] <icy> there are examples for both sides
[Saturday 11 September 2010] [08:42:35] <icy> pieter_hintjens: uhm.. ;)
[Saturday 11 September 2010] [08:42:40] <pieter_hintjens> :-)
[Saturday 11 September 2010] [08:43:03] <guido_g> icy: but it doesn’t work that way
[Saturday 11 September 2010] [08:43:10] <pieter_hintjens> anyhow, this is why 0MQ very carefully emulates a poll function everyone hates but totally trusts and understands
[Saturday 11 September 2010] [08:44:12] <icy> guido_g: just curious, why was that lib no fun?
[Saturday 11 September 2010] [08:44:25] <guido_g> icy: wllm
[Saturday 11 September 2010] [08:44:39] <icy> pieter_hintjens: I totally see your point
[Saturday 11 September 2010] [08:45:02] <icy> wllm?
[Saturday 11 September 2010] [08:45:09] <pieter_hintjens> icy: once there is a significant weight behind 0MQ (and we’re getting there) we can be more ambitious about our own abstractions
[Saturday 11 September 2010] [08:45:41] <guido_g> icy: from a small 3 letter company (HAL) :)
[Saturday 11 September 2010] [08:46:12] <icy> pieter_hintjens: wouldn’t it be hard to make a radical api change once you got a considerable amount of applications using it? as a counter argument
[Saturday 11 September 2010] [08:46:17] <icy> guido_g: will take a look, thx
[Saturday 11 September 2010] [08:46:21] <pieter_hintjens> icy: it’s about layers
[Saturday 11 September 2010] [08:46:28] <guido_g> icy: no need to, really :)
[Saturday 11 September 2010] [08:46:31] <pieter_hintjens> we’d never make radical api changes
[Saturday 11 September 2010] [08:46:41] <pieter_hintjens> just add layers on top
[Saturday 11 September 2010] [08:47:14] <pieter_hintjens> e.g. event driven frameworks that actually call the poll API
[Saturday 11 September 2010] [08:47:32] <pieter_hintjens> though I suspect poll API itself will be redesigned before the 0MQ is totally stabilized
[Saturday 11 September 2010] [08:47:40] <guido_g> so the developer can pick his poison depending on the use-case etc.
[Saturday 11 September 2010] [08:52:10] <pieter_hintjens> yeah
[Saturday 11 September 2010] [08:52:25] <icy> for the non blocking part I’d argue it makes it even easier given the atomicity of messages
[Saturday 11 September 2010] [08:53:50] <guido_g> we had a lot of trouble w/ these callbacks
[Saturday 11 September 2010] [08:54:32] <guido_g> one problem is that you are still in the library
[Saturday 11 September 2010] [08:54:45] <guido_g> which leads to more or less well known constraints
[Saturday 11 September 2010] [08:55:06] <pieter_hintjens> threading issues etc.
[Saturday 11 September 2010] [08:55:07] <guido_g> i wouldn’t go that route again
[Saturday 11 September 2010] [08:55:15] <pieter_hintjens> the reactor pattern looks more sane
[Saturday 11 September 2010] [08:55:17] <guido_g> pieter_hintjens: oh yes!
[Saturday 11 September 2010] [08:55:27] <guido_g> to both
[Saturday 11 September 2010] [08:55:36] <pieter_hintjens> what i’d love to make is a Libero/0MQ at some stage
[Saturday 11 September 2010] [08:56:04] <guido_g> sorry, what would that be?
[Saturday 11 September 2010] [08:56:08] <pieter_hintjens> well, something like that, anyhow: state machines where events are 0MQ messages
[Saturday 11 September 2010] [08:56:42] <pieter_hintjens> you turn the loop inside out, the loop calls your code and feeds it incoming messages as events
[Saturday 11 September 2010] [08:56:49] <pieter_hintjens> and your code goes through states
[Saturday 11 September 2010] [08:56:57] <guido_g> makes sense
[Saturday 11 September 2010] [08:57:13] <pieter_hintjens> it’s a pattern that can be terribly useful but it can be hard to grok as well
[Saturday 11 September 2010] [08:57:33] <pieter_hintjens> only worth doing for more complex 0MQ apps but I suspect we’re heading that way
[Saturday 11 September 2010] [08:57:46] <pieter_hintjens> devices that manage 3+ sockets
[Saturday 11 September 2010] [08:57:46] <guido_g> sure
[Saturday 11 September 2010] [08:59:52] <icy> yes this is what I was refering to. you give a socket a function to call upon receiving a message. this way you don’t need to care about searching which sockets got a message after a poll
[Saturday 11 September 2010] [09:00:12] <icy> zmq would do it for you and can therefor use the faster implementation possible for a given platform
[Saturday 11 September 2010] [09:00:29] <pieter_hintjens> icy: you can trivially build this on top
[Saturday 11 September 2010] [09:00:36] <pieter_hintjens> perhaps a fun thing to make
[Saturday 11 September 2010] [09:00:59] <guido_g> icy: as i said, it wasn’t nice to use in a real world app
[Saturday 11 September 2010] [09:01:08] <pieter_hintjens> i can see several problems with it, though
[Saturday 11 September 2010] [09:01:32] <guido_g> plus the problems comming from the product itself
[Saturday 11 September 2010] [09:01:33] <pieter_hintjens> passing control to a function means you can’t e.g. work with multiple sockets at once
[Saturday 11 September 2010] [09:01:45] <pieter_hintjens> so you can’t do stuff like devices that read from one, write to the other
[Saturday 11 September 2010] [09:01:58] <pieter_hintjens> or you need to create and pass around increasingly complex contexts
[Saturday 11 September 2010] [09:01:58] <guido_g> you can’t even do a proper error handling :)
[Saturday 11 September 2010] [09:02:19] <guido_g> which then slow you down even more
[Saturday 11 September 2010] [09:02:24] <pieter_hintjens> but it could be fun to experiment with
[Saturday 11 September 2010] [09:02:48] <pieter_hintjens> icy: take a look at the various reactors people like Matt Weinstein and Chuck Remes have made
[Saturday 11 September 2010] [09:04:08] <icy> of course you can do proper error handling. maybe your experience was because of a poor product, not the methodology itself guido_g
[Saturday 11 September 2010] [09:04:32] <icy> but yea to handle multiple sockets you need to pass around also some additional data
[Saturday 11 September 2010] [09:05:05] <guido_g> callbacks are generally not the first choice, i did X11 and Motif and learned to avoid them
[Saturday 11 September 2010] [09:05:16] <guido_g> then wllm, smame thing
[Saturday 11 September 2010] [09:05:21] <guido_g>
*same
[Saturday 11 September 2010] [09:05:51] <guido_g> if you want it, you should do it in the app
[Saturday 11 September 2010] [09:06:31] <guido_g> libs or frameworks can’t know enough of the use-cases to provide a proper solution for every user
[Saturday 11 September 2010] [09:06:58] <guido_g> this is why frameworks tend to become so big and bloated
[Saturday 11 September 2010] [09:09:01] <pieter_hintjens> icy: what i’ve found with 0MQ is that theory is pretty useless :-)
[Saturday 11 September 2010] [09:09:12] <pieter_hintjens> get your hands dirty with code and it becomes rapidly clear
[Saturday 11 September 2010] [09:09:24] <pieter_hintjens> happily what you learn is the same no matter what language you use
[Saturday 11 September 2010] [11:11:05] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 11 September 2010] [11:58:53] Quit keffo has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [13:08:35] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [13:14:50] Quit gebi has left this server (Read error: Operation timed out).
[Saturday 11 September 2010] [13:24:13] <keffo> um, push/pull is not pub/sub, or did i confuse things?
[Saturday 11 September 2010] [13:25:01] <keffo> ah right
[Saturday 11 September 2010] [13:37:30] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [13:58:22] Join old_sound has joined this channel (
~mrhyde@222.44.41.33).
[Saturday 11 September 2010] [13:59:01] Part old_sound has left this channel.
[Saturday 11 September 2010] [14:01:25] Quit xla has left this server (Ping timeout: 260 seconds).
[Saturday 11 September 2010] [14:11:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [14:14:38] Quit rbraley has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [15:04:18] <iFire> I mean is it feasible
*** on Sat Sep 11 16:30:13 2010
[Saturday 11 September 2010] [16:30:13] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [16:30:19] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [16:30:19] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [17:22:47] Quit rgl has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [18:04:34] <psino> is there any way to tell a socket to stop connecting to an endpoint, or stop listening on an endpoint?
[Saturday 11 September 2010] [18:19:59] <psino> and is there a way of gracefully shutting down a worker process that uses an incoming PULL socket?
[Saturday 11 September 2010] [18:37:44] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 11 September 2010] [18:53:43] Quit troutwine has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [19:06:05] Quit plq has left this server (Ping timeout: 258 seconds).
[Saturday 11 September 2010] [23:22:58] Quit rgl has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [23:33:09] Quit kooroo has left this server (Quit: HydraIRC ->
http://www.hydrairc.com <-).
[Sunday 12 September 2010] [00:22:53] Quit Samy has left this server (
*.net
*.split).
[Sunday 12 September 2010] [00:22:54] Quit mivert has left this server (
*.net
*.split).
[Sunday 12 September 2010] [01:01:42] Quit Samy has left this server (Excess Flood).
[Sunday 12 September 2010] [01:01:43] Quit rbraley has left this server (Excess Flood).
[Sunday 12 September 2010] [01:02:03] Quit soren has left this server (Remote host closed the connection).
[Sunday 12 September 2010] [01:18:59] Quit Tasser has left this server (
*.net
*.split).
[Sunday 12 September 2010] [01:42:38] Quit beppu has left this server (Ping timeout: 245 seconds).
[Sunday 12 September 2010] [01:49:53] Quit xla has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [02:12:18] Quit jsimmons has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [03:36:03] <pieter_hintjens> iFire: I’m going to make an example of that in Ch3
[Sunday 12 September 2010] [03:36:06] <pieter_hintjens> of the guide
*** on Sun Sep 12 03:46:31 2010
[Sunday 12 September 2010] [03:46:31] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [03:46:37] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [03:46:37] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [03:49:06] <iFire> pieter_hintjens chapater ch3 where
[Sunday 12 September 2010] [03:50:47] <iFire> where will the chapter 3 be pieter_hintjens?
[Sunday 12 September 2010] [03:51:20] <guido_g> user guid
[Sunday 12 September 2010] [03:51:22] <guido_g> +e
[Sunday 12 September 2010] [07:53:47] Quit psino has left this server (Ping timeout: 255 seconds).
[Sunday 12 September 2010] [08:23:28] Quit rgl has left this server (Ping timeout: 255 seconds).
[Sunday 12 September 2010] [09:12:21] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [10:14:10] Quit psino has left this server (Quit: psino).
[Sunday 12 September 2010] [10:45:27] Quit omarkj has left this server (Quit: omarkj).
[Sunday 12 September 2010] [11:44:45] Quit
benoitc_ has left this server (Changing host).
[Sunday 12 September 2010] [11:45:00] Nick
benoitc_ is now known as benoitc.
[Sunday 12 September 2010] [12:58:50] Quit jsimmons has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [13:22:40] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [13:27:37] Quit
keffo_ has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [13:47:41] <mankins> messages can be of arbitrary length, right?
[Sunday 12 September 2010] [13:56:35] <icy> they have to fit in ram
[Sunday 12 September 2010] [14:21:18] Quit gebi has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [14:31:50] Quit
gebi_ has left this server (Ping timeout: 264 seconds).
[Sunday 12 September 2010] [14:49:05] <mankins> i’m having some issue with messages getting corrupted on the wire.
[Sunday 12 September 2010] [14:49:29] <mankins> trying to isolate it.
[Sunday 12 September 2010] [14:49:54] <mankins> there are known issues with 2.0.9, are there? (I checked the bug list, and nothing jumped out)
[Sunday 12 September 2010] [14:52:35] Quit gebi has left this server (Ping timeout: 252 seconds).
[Sunday 12 September 2010] [15:11:32] Quit gebi has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [15:17:39] <mankins> hrm, maybe it was the perl serializer.
[Sunday 12 September 2010] [15:18:14] <mankins> i was using send/recv but changed to send_as/recv and it seems to be better. <shrug>
[Sunday 12 September 2010] [15:34:31] Quit gebi has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [15:44:10] Quit rgl has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [15:53:05] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [15:54:10] Quit
keffo__ has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [16:00:02] Quit gebi has left this server (Ping timeout: 264 seconds).
[Sunday 12 September 2010] [16:03:28] Quit mankins has left this server (Quit: mankins).
[Sunday 12 September 2010] [16:19:41] Quit rgl has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [16:32:54] Quit
soren_ has left this server (Changing host).
[Sunday 12 September 2010] [16:33:44] Nick
soren_ is now known as soren.
[Sunday 12 September 2010] [16:45:56] Quit plq has left this server (Quit: Leaving).
[Sunday 12 September 2010] [17:11:53] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [17:39:22] Part bobs has left this channel.
[Sunday 12 September 2010] [18:28:18] <kenkeiter> Looking for some input on a decentralized pub-sub/dequeue model with zeromq
[Sunday 12 September 2010] [18:30:23] <kenkeiter> need to be able to publish messages in a specific category, subscribe to that category and (hopefully) pull jobs from it in a load-balanced way.
[Sunday 12 September 2010] [18:30:28] <kenkeiter> Any suggestions?
[Sunday 12 September 2010] [19:26:18] Quit
keffo__ has left this server (Ping timeout: 272 seconds).
[Sunday 12 September 2010] [19:26:22] <sleeperbot> Hey guys, wondering if I could get some idea on what’s going on with my installation of zmq
[Sunday 12 September 2010] [19:26:39] <sleeperbot> been using zeromq and my CPU usage shot up x100. I’m doing Upstream/Downstream, using the node.js wrapper.
[Sunday 12 September 2010] [19:26:46] <sleeperbot> Leaving connections open on both ends to send and receive
[Sunday 12 September 2010] [19:32:15] <kenkeiter> Don’t know a tremendous amount about zmq’s internals, but my suggestion would be to verify that you have the correct number of zmq threads spawned.
[Sunday 12 September 2010] [19:32:49] <kenkeiter> If you’re respawning for each connection, they may also not be released properly..
[Sunday 12 September 2010] [19:32:57] <kenkeiter> Which could pose another problem.
*** on Sun Sep 12 19:37:10 2010
[Sunday 12 September 2010] [19:37:10] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [19:37:16] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [19:37:16] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [19:43:52] Quit andrewvc has left this server (Ping timeout: 272 seconds).
[Sunday 12 September 2010] [19:53:57] <sleeperbot> Thanks Ken, i’ll check
[Sunday 12 September 2010] [19:54:21] <sleeperbot> I’m not respawning so it could be the node.js driver
[Sunday 12 September 2010] [19:59:18] Quit GeekGod has left this server (Quit: GeekGod).
[Sunday 12 September 2010] [20:00:10] <kenkeiter> sleeperbot: it was worth a try.. did you verify that you’re only running one or two messaging threads?
[Sunday 12 September 2010] [20:00:13] <andrewvc> I’ve been curious as to how stable the node driver is
[Sunday 12 September 2010] [20:00:20] <kenkeiter> :/
[Sunday 12 September 2010] [20:03:14] <sleeperbot> I’m looking up how to do that
[Sunday 12 September 2010] [20:03:52] <sleeperbot> do you know what I can type in the command line to bring that info up?
[Sunday 12 September 2010] [20:04:03] <kenkeiter> sleeperbot: which platform?
[Sunday 12 September 2010] [20:04:25] <sleeperbot> unix
[Sunday 12 September 2010] [20:04:30] <sleeperbot> ubuntu karmic
[Sunday 12 September 2010] [20:07:43] <kenkeiter> htop might work.. haven’t done it under
*ni
[Sunday 12 September 2010] [20:14:13] <sleeperbot> I see 3 versions of my node.js stream and web servers
[Sunday 12 September 2010] [20:14:27] <sleeperbot> don’t see anything related to zmq
[Sunday 12 September 2010] [20:15:25] <sleeperbot> killed the extraneous processes, will check if anything changed in cpu usage
*** on Sun Sep 12 20:17:38 2010
[Sunday 12 September 2010] [20:17:38] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [20:17:43] Part andrewvc has left this channel.
[Sunday 12 September 2010] [20:17:44] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [20:17:44] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [20:21:22] Quit bennymack has left this server (Ping timeout: 252 seconds).
[Sunday 12 September 2010] [20:33:24] Quit psino has left this server (Quit: psino).
[Sunday 12 September 2010] [20:39:02] Quit jsimmons has left this server (Ping timeout: 264 seconds).
*** on Sun Sep 12 21:09:03 2010
[Sunday 12 September 2010] [21:09:03] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [21:09:09] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [21:09:09] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [22:57:18] Part pieter_hintjens has left this channel.
[Sunday 12 September 2010] [23:25:32] Quit kenkeiter has left this server (Quit: kenkeiter).
[Sunday 12 September 2010] [23:34:09] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [23:34:24] <andrewvc> I assume that XREQ/XREP sockets apply backpressure in the same manner as PUSH/PULL and REQ/REP yes?
[Sunday 12 September 2010] [23:59:40] Quit sleeperbot has left this server (Ping timeout: 252 seconds).
[Monday 13 September 2010] [00:15:34] Quit xla has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [00:44:16] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [00:56:55] Quit bbigras has left this server (Ping timeout: 252 seconds).
[Monday 13 September 2010] [01:30:06] Quit guido_g has left this server (Quit: Boom...).
[Monday 13 September 2010] [01:53:45] Quit
keffo__ has left this server (Read error: Connection reset by peer).
*** on Mon Sep 13 02:24:03 2010
[Monday 13 September 2010] [02:24:03] Topic The channel topic is “Welcome!”.
[Monday 13 September 2010] [02:24:09] Mode Channel modes: no messages from outside, topic protection
[Monday 13 September 2010] [02:24:09] Created This channel was created on 2010-02-05 10:44.
[Monday 13 September 2010] [02:36:59] Quit rgl has left this server (Ping timeout: 255 seconds).
[Monday 13 September 2010] [02:47:16] Quit kenkeiter has left this server (Quit: kenkeiter).
[Monday 13 September 2010] [03:18:13] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [04:37:41] Join plq has joined this channel (
~plq@78.189.30.218).
[Monday 13 September 2010] [05:08:18] Part keffo has left this channel (“Leaving”).
[Monday 13 September 2010] [07:09:02] Quit omarkj has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [07:09:03] Nick
omarkj_ is now known as omarkj.
[Monday 13 September 2010] [07:28:04] <CIA-20> zeromq2: 03Martin Lucina 07master * rbe159b6 10/ src/pipe.cpp : zmq::writer_t: Add missing test for swap - http://bit.ly/aGN4bM
[Monday 13 September 2010] [07:29:19] <icy> sustrik: hi, is there any paper on the algorithm used for the lock-free queue?
[Monday 13 September 2010] [07:30:51] <sustrik> icy: there’s a very old article here:
[Monday 13 September 2010] [07:30:59] <sustrik> lot of it doesn’t apply any more
[Monday 13 September 2010] [07:31:42] <sustrik> this is what still applies: “+ Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> - Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> FoldUnfold
[Monday 13 September 2010] [07:31:42] <sustrik> Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> Introduction
[Monday 13 September 2010] [07:31:44] <sustrik> Design
[Monday 13 September 2010] [07:31:46] <sustrik> Performance
[Monday 13 September 2010] [07:31:48] <sustrik> Configuration
[Monday 13 September 2010] [07:31:50] <sustrik> Performance with polling
[Monday 13 September 2010] [07:31:52] <sustrik> Performance without polling
[Monday 13 September 2010] [07:31:54] <sustrik> Conclusion
[Monday 13 September 2010] [07:31:56] <sustrik> HISTORICAL WHITEPAPER
[Monday 13 September 2010] [07:31:58] <sustrik> Introduction
[Monday 13 September 2010] [07:32:00] <sustrik> Y-suite is a set of components designed for ultra-efficient passing of messages between threads within a process. Y-suite is somehow similar to local sockets, however, it is much faster.
[Monday 13 September 2010] [07:32:03] <sustrik> In version 0.1 of MQ lightweight messaging kernel, the only y-suite component available is ypipe, a lock-free and wait-free implementation of a queue. In version 0.2 ypollset is added to allow thread to interchange messages with several other threads at the same time (similar to POSIX poll function). Component known as semaphore in version 0.1 is renamed to ysemaphore in version 0.2 to mark that it belongs to y-suite. Same way, spipe i
[Monday 13 September 2010] [07:32:08] <sustrik> s renamed to ysocketpair.
[Monday 13 September 2010] [07:32:10] <sustrik> Design
[Monday 13 September 2010] [07:32:14] <sustrik> The basic means of transferring message between threads is ypipe. Messages are passed through a pipe in the standard write and read manner. Once the reader has no more messages to read from the pipe, it notifies the sender using passive synchronization and goes asleep. Passive synchronization means that the other thread is not notified directly using some kind of async signal, rather it will be notified once it tries to write the next me
[Monday 13 September 2010] [07:32:19] <sustrik> ssage to the pipe. When this happens, writer becomes aware that reader is already asleep or at least going asleep at the moment. It knows that there is new message available, so it wakes the reader up using active synchronization, i.e. actively sending wake-up event to the other thread. Active synchronisation is not provided by ypipe itself, rather by other y-suite components, to be discussed bellow. Usage of ypipe is depicted on the fol
[Monday 13 September 2010] [07:32:24] <sustrik> lowing sequence diagram:”
[Monday 13 September 2010] [07:32:26] <sustrik> yuck
[Monday 13 September 2010] [07:32:28] <sustrik> sorry
[Monday 13 September 2010] [07:33:01] <sustrik> too much text, but the last paragraph is relevant
[Monday 13 September 2010] [07:33:36] <sustrik> also see the diagram that follows the text above
[Monday 13 September 2010] [07:34:23] <ekidd> Good morning! ZeroMQ is a really nice library.
[Monday 13 September 2010] [07:35:08] <ekidd> If I’m using REQ/REP messaging with multiple servers, what happens if one server is asked to handle an unusually long-running request?
[Monday 13 September 2010] [07:36:29] <ekidd> Do the clients just route requests to one of the available servers? Or do they continue to send requests to the busy server?
[Monday 13 September 2010] [07:36:30] <sustrik> ekidd: if you set high watermark, it’s queue gets eventually full and subsequent requests will be dispatches to other servers
[Monday 13 September 2010] [07:36:59] <icy> sustrik: yea I’ve read that, I guess because it is single-reader single-write, it does not suffer from the ABA problem?
[Monday 13 September 2010] [07:37:38] <ekidd> sustrik: Ah, OK. The useful high watermark in my case is very small: The servers are inherently single-threaded workers with long-running jobs. I want to keep them loaded.
[Monday 13 September 2010] [07:37:45] <sustrik> icy: what’s ABA?
[Monday 13 September 2010] [07:38:02] <ekidd> I do, however, have lots of clients and servers.
[Monday 13 September 2010] [07:39:11] <icy> sustrik: it’s one of the main problems that lock-free queues have to overcome
[Monday 13 September 2010] [07:39:23] <guido_g> it takes an unusually long time to complete the request
[Monday 13 September 2010] [07:40:01] <ekidd> icy: My clients and servers are different machines, so I don’t think the lock-free stuff is relevant. But I might be confused.
[Monday 13 September 2010] [07:40:29] <sustrik> ekidd: that’s a different conversion going on :)
[Monday 13 September 2010] [07:40:40] <ekidd> Ah, OK. I was confused. :-)
[Monday 13 September 2010] [07:41:02] <guido_g> ekidd: did you see that req/req is locked to the send/recv order?
[Monday 13 September 2010] [07:41:02] <sustrik> ekidd: there’s no such thing in 0MQ as explicit ack
[Monday 13 September 2010] [07:41:27] <sustrik> so there’s no way for it to work in lock-step fashion
[Monday 13 September 2010] [07:41:32] <ekidd> guido_g: Yeah, that works for me.
[Monday 13 September 2010] [07:42:32] <sustrik> icy: it’s basically a two step process
[Monday 13 September 2010] [07:42:32] <guido_g> same for the rep side (the server)
[Monday 13 September 2010] [07:42:41] <ekidd> I basically have a farm of Windows workers that take 0.1 to (say) 60 seconds to process a job, and idle time costs money. There’s one worker per server.
[Monday 13 September 2010] [07:42:46] <guido_g> see the user guide for lots of examples and ideas
[Monday 13 September 2010] [07:43:12] <sustrik> icy: while there are messages to be read the synchronisation is done simply by moving a pointer in the linked list in atomic manner
[Monday 13 September 2010] [07:43:32] <sustrik> icy: when there are no messages to be read, the pointer becomes NULL
[Monday 13 September 2010] [07:43:32] <ekidd> I don’t mind locking the order of responses the way req/rep does: I’m talking to expensive, single-threaded Windows libraries in any case.
[Monday 13 September 2010] [07:44:01] <icy> sustrik: understood so far and it seems it does not suffer from aba, just was curious if there was real proof of the correctness of the algorithm
[Monday 13 September 2010] [07:44:05] <sustrik> icy: reader goes asleep and standard inter-thread mechanism (socketpair) is used to wake it up
[Monday 13 September 2010] [07:44:16] <ekidd> But I want to maximize utilization of those expensive libraries.
[Monday 13 September 2010] [07:44:23] <sustrik> icy: no
[Monday 13 September 2010] [07:44:29] <sustrik> want to prove it?
[Monday 13 September 2010] [07:44:46] <icy> that would take more time than I have probably :)
[Monday 13 September 2010] [07:45:08] <ekidd> As long as zeromq clients respect the individual server’s high water marks and route requests to another worker, everything will work fine.
[Monday 13 September 2010] [07:46:09] <ekidd> I’m going to write some tests (of course). I just wanted to know whether I was even trying something sane. :-)
[Monday 13 September 2010] [07:48:01] <ekidd> Many thanks for your advice, folks!
[Monday 13 September 2010] [07:54:15] <ekidd> guido_g: Excellent. It definitely has the right semantics. I’ll still need to find out whether it does the right thing, performance-wise, with large messages and queues that are often at their high water marks.
[Monday 13 September 2010] [08:37:13] Quit rgl has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [08:39:00] Part gebi has left this channel.
[Monday 13 September 2010] [08:51:31] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [09:27:32] Quit plq has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [09:33:25] Quit guido_g has left this server (Quit: Boom...).
[Monday 13 September 2010] [09:43:13] Quit rbraley has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [09:44:53] <cremes> while writing some specs for my bindings this weekend i came across a few issues with SWAP, RECOVERY_IVL and RATE
[Monday 13 September 2010] [09:45:08] <cremes> all 3 of those take signed 64-bit integers for input
[Monday 13 September 2010] [09:45:32] <cremes> they also do not return an error when passed a negative number even though that doesn’t make any sense
[Monday 13 September 2010] [09:45:56] <cremes> should the library return an error for negative numbers or should my bindings take care of that issue?
[Monday 13 September 2010] [09:49:52] Join plq has joined this channel (
~plq@78.189.30.218).
[Monday 13 September 2010] [09:55:33] Quit jsimmons has left this server (Ping timeout: 245 seconds).
[Monday 13 September 2010] [09:58:11] <ptrb> I start the server, it sits at zmq_recv(), great; i run the client, it runs fine and exits, but the server never receives anything. ideas?
[Monday 13 September 2010] [10:00:05] <pieterh> cremes: i think all the setsockopt types need to be reviewed for 3.0
[Monday 13 September 2010] [10:00:41] <cremes> pieterh: ok; so should i open bugs for those against the 2.1.x branch?
[Monday 13 September 2010] [10:00:42] <pieterh> but certainly if they are signed get a negative value that should return EINVAL
[Monday 13 September 2010] [10:00:51] <pieterh> yup, even 2.0.x IMO
[Monday 13 September 2010] [10:00:57] <cremes> ok, i’ll do that now
[Monday 13 September 2010] [10:01:21] <pieterh> ptrb: looking at it...
[Monday 13 September 2010] [10:02:01] <ptrb> pieterh: thx; I’m guessing there’s some setup step I’ve overlooked
[Monday 13 September 2010] [10:02:23] <pieterh> ptrb: try ‘ps’
[Monday 13 September 2010] [10:02:31] <pieterh> imo you have a second copy of the server running
[Monday 13 September 2010] [10:02:52] <pieterh> (though it would assert then...)
[Monday 13 September 2010] [10:03:11] <pieterh> sorry, forget I said that plz
[Monday 13 September 2010] [10:03:11] <ptrb> hmm, no, but maybe something else is sitting on 5001, let me try changing that
[Monday 13 September 2010] [10:03:22] <pieterh> ptrb: client writes a message and then closes & exits
[Monday 13 September 2010] [10:03:28] <pieterh> two things: (a) it should wait for a reply
[Monday 13 September 2010] [10:03:37] <pieterh> (b) if it does not want to wait, it can’t exit immediately
[Monday 13 September 2010] [10:03:44] <pieterh> you need to read the users guide
[Monday 13 September 2010] [10:04:00] <ptrb> I have.
[Monday 13 September 2010] [10:04:10] <pieterh> 0mq/2.0.x loses data if you close the socket while there is data in flight
[Monday 13 September 2010] [10:04:11] <cremes> ptrb: are you starting the server first?
[Monday 13 September 2010] [10:04:16] <ptrb> Of course.
[Monday 13 September 2010] [10:04:23] <pieterh> send/close is not going to work
[Monday 13 September 2010] [10:04:28] <pieterh> send/recv/close is ok
[Monday 13 September 2010] [10:04:32] <pieterh> send/sleep/close is ok
[Monday 13 September 2010] [10:04:32] <ptrb> OK, so, do I need to recv() in the clie... k
[Monday 13 September 2010] [10:04:34] <cremes> ah yes, that’s right
[Monday 13 September 2010] [10:04:43] <cremes> do a sleep before exiting
[Monday 13 September 2010] [10:04:45] <ptrb> even if I don’t post anything back explicitly?
[Monday 13 September 2010] [10:04:47] <pieterh> prtb: either a recv
[Monday 13 September 2010] [10:04:59] Nick
bbigras_ is now known as bbigras.
[Monday 13 September 2010] [10:05:07] <pieterh> prtb if you’re using REQ and REP sockets, you should be doing send/recv and recv/send
[Monday 13 September 2010] [10:05:20] <pieterh> if you want to just send 1 message as such use PUSH/PULL
[Monday 13 September 2010] [10:05:39] <pieterh> it’s not a biggie
[Monday 13 September 2010] [10:05:52] <pieterh> the problem here is not giving the client process time to send its data
[Monday 13 September 2010] [10:05:53] <ptrb> I’m doing something vaguely RPC-ish, so I guess if I want to represent a void blah(); I still have to send something back
[Monday 13 September 2010] [10:06:06] <pieterh> or else use XREQ/XREP
[Monday 13 September 2010] [10:06:09] <ptrb> yeah it makes sense, sure. thanks. i guess it’s just not explicit anywhere in the docs (afaict)
[Monday 13 September 2010] [10:06:18] <pieterh> rtfug... :-)
[Monday 13 September 2010] [10:06:22] <pieterh> it is explicit in there
[Monday 13 September 2010] [10:06:31] <ptrb> i have; if you want to point me to the sentence in question I’m happy to be made a fool
[Monday 13 September 2010] [10:06:46] * pieterh goes off to search for ‘sleep’...
[Monday 13 September 2010] [10:07:11] <pieterh> Note that we do sleep (1); before exiting the ventilator. This is a hack that gets around MQ/2.0’s design, which discards messages that have not yet been sent, if you exit the program too soon. If you are using MQ/2.1 you can remove this sleep statement.
[Monday 13 September 2010] [10:07:43] <ptrb> eh.
[Monday 13 September 2010] [10:08:19] <pieterh> it’s the first example that has this problem, so I explain it there
[Monday 13 September 2010] [10:08:29] <pieterh> the hello world client waits for an answer
[Monday 13 September 2010] [10:08:38] <pieterh> and the pubsub example never exits
[Monday 13 September 2010] [10:09:01] <pieterh> maybe i should put it in bold...
[Monday 13 September 2010] [10:09:13] <pieterh> and repeat this, it’s a common fault
[Monday 13 September 2010] [10:11:29] <ptrb> if you’re willing to take some constructive criticism about the documentation, i’d say that while example-based docs are great, when I have a specific problem (like this) I find there isn’t really a way I can find a solution; there’s no idioms or implementation details or whatever to search through (as far as I’ve found)
[Monday 13 September 2010] [10:13:56] <ptrb> but!! but but, thank you :)
[Monday 13 September 2010] [10:19:21] <ptrb> hmm, recv on the client side hangs... is there not some zmq_flush or something I can call?
[Monday 13 September 2010] [10:20:33] <guido_g> no
[Monday 13 September 2010] [10:20:40] <ptrb> poop :|
[Monday 13 September 2010] [10:21:06] <guido_g> pardon?
[Monday 13 September 2010] [10:21:19] <ptrb> that was an expression of mild disappointment
[Monday 13 September 2010] [10:31:20] Quit plq has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [10:40:40] Quit mato has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [11:04:45] Quit rbraley has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [11:16:08] Quit mivert has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [11:16:51] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [11:35:55] <pieterh> ptrb: still there?
[Monday 13 September 2010] [11:36:04] <pieterh> sorry, was in a meeting
[Monday 13 September 2010] [11:36:19] <ptrb> yeah sure
[Monday 13 September 2010] [11:36:54] <pieterh> making a problem driven section in the guide would be good
[Monday 13 September 2010] [11:37:01] <pieterh> did you find out why your client hangs?
[Monday 13 September 2010] [11:37:29] <ptrb> No, I just threw a sleep in there and moved on to bigger, even more problematic things :)
[Monday 13 September 2010] [11:37:41] <ptrb> a problem-driven section would be good, but it’ll never be comprehensive
[Monday 13 September 2010] [11:38:09] <ptrb> FWIW I think a good documentation model would be ZeroC’s ICE, which has a really comprehensive .pdf
[Monday 13 September 2010] [11:38:16] <pieterh> “did not get a message” is a pretty classic stumbling block
[Monday 13 September 2010] [11:38:23] <ptrb> yeah, fair
[Monday 13 September 2010] [11:38:33] <pieterh> i’ll write a flowchart
[Monday 13 September 2010] [11:40:41] <ptrb> now, i’m working on an implementation based on the multithreaded code in the user guide, and i’m getting infinite size-0 messages on the server side after sending one legitimate message from a client
[Monday 13 September 2010] [11:40:48] <ptrb> ever hear of something like this?
[Monday 13 September 2010] [11:42:33] <ptrb> sorry, based on the multithreaded server in the introduction doc
[Monday 13 September 2010] [11:43:41] <cremes> ptrb: i’ve never seen that... you say your code is “based on” the example; it’s always a good idea to start from code that you know works and modify from there
[Monday 13 September 2010] [11:43:49] <cremes> sounds like your mods broke it
[Monday 13 September 2010] [11:44:08] <cremes> the easiest way to find the failure is to revert back to the original “good” code and slowly modify it to your specifications
[Monday 13 September 2010] [11:44:27] <ptrb> yeah. i know. i’m trying to drop the server into an existing process to provide a zmq “layer”, so there’s not really any way to iterate my way to where I am now.
[Monday 13 September 2010] [11:45:07] <ptrb> i guess i can try taking out some functionality.
[Monday 13 September 2010] [11:45:20] <cremes> did you change the code that sends 0mq messages?
[Monday 13 September 2010] [11:48:35] <ptrb> yes; in ways i initially thought were inconsequential, but i suppose i’m in an assumption-revalidating mood :)
[Monday 13 September 2010] [11:49:15] <ptrb> as a meta-comment, it’s really great you guys are hanging out on irc to help folks; zmq is a great project and this is a great resource.
[Monday 13 September 2010] [11:50:54] <ptrb> aha! so, if i zmq_recv(), get a message, and don’t zmq_send() something in the server, subsequent zmq_recv()s have the effect of not blocking
[Monday 13 September 2010] [11:51:09] <ptrb> ...which seems quite strange to me
[Monday 13 September 2010] [11:51:57] <cremes> ptrb: what kind of socket are you using on this server side?
[Monday 13 September 2010] [11:53:06] <cremes> because that behavior doesn’t sound right; the zmq_recv() call is returning an error, right?
[Monday 13 September 2010] [11:54:04] <ptrb> yeah, it returns -1 EAGAIN
[Monday 13 September 2010] [11:54:11] <ptrb> i believe it’s EAGAIN, at least.
[Monday 13 September 2010] [11:56:31] <ptrb> the topology is the multithread server example in the intro doc: public tcp XREP endpoint, managed by one thread running zmq_device(ZMQ_QUEUE, ...), forwarding via XREQ to an inproc endpoint, being consumed by worker threads binding to REP
[Monday 13 September 2010] [12:02:56] <cremes> can you provide a code pastie?
[Monday 13 September 2010] [12:03:14] <ptrb> it won’t be complete, but sure, one sec...
[Monday 13 September 2010] [12:03:38] <cremes> it doesn’t need to be complete... i want to see the code that sets up the socket and calls recv on it
[Monday 13 September 2010] [12:04:03] <ptrb> the worker thread ultimately responsible for processing the recv, right?
[Monday 13 September 2010] [12:04:39] <cremes> whatever code is returning -1 EAGAIN
[Monday 13 September 2010] [12:06:16] Quit xla has left this server (Quit: leaving).
[Monday 13 September 2010] [12:08:10] <cremes> ptrb: in your DEBUG statement, also print out the value of zmq_strerror()
[Monday 13 September 2010] [12:08:27] <cremes> i need more information to figure this out
[Monday 13 September 2010] [12:09:15] Quit xla has left this server (Client Quit).
[Monday 13 September 2010] [12:09:18] <ptrb> Operation cannot be accomplished in current state
[Monday 13 September 2010] [12:10:02] <cremes> ah, then there we have it; with a REP socket you can’t call recv again until you have subsequently called send
[Monday 13 September 2010] [12:10:16] <cremes> it needs that recv/send/recv pattern because it maintains a small internal state machine
[Monday 13 September 2010] [12:10:23] <ptrb> oh, interesting
[Monday 13 September 2010] [12:10:24] <cremes> that’s the whole point of the REQ/REP socket pattern
[Monday 13 September 2010] [12:10:24] <ptrb> ok
[Monday 13 September 2010] [12:10:34] <ptrb> see, this is useful! this should be on a website somewhere :)
[Monday 13 September 2010] [12:10:40] <cremes> the worker is supposed to respond when it is done, right?
[Monday 13 September 2010] [12:10:50] <ptrb> well, my thought is that it may optionally respond
[Monday 13 September 2010] [12:10:51] <cremes> it is for sure
[Monday 13 September 2010] [12:10:58] <ptrb> but if it has to respond, that’s fine too
[Monday 13 September 2010] [12:11:07] <cremes> if you want it to be optional, use XREP sockets
[Monday 13 September 2010] [12:11:25] <cremes> that kind of socket does not enforce the recv/send/recv pattern
[Monday 13 September 2010] [12:11:45] <ptrb> I suspect there is more to XREP than simply dropping that enforcement, though
[Monday 13 September 2010] [12:12:53] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [12:13:51] <cremes> ptrb: not really; REP sockets are built on top of the XREP socket
[Monday 13 September 2010] [12:14:00] <ptrb> hmm, interesting
[Monday 13 September 2010] [12:14:14] <cremes> REP sockets know how to “route” their responses back over multiple hops
[Monday 13 September 2010] [12:14:29] <cremes> you need to do a little extra work when using an XREP socket to retain that functionality
[Monday 13 September 2010] [12:18:53] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [12:26:08] <ptrb> curiouser and curiouser
[Monday 13 September 2010] [12:26:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [12:39:20] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Monday 13 September 2010] [13:10:03] Quit bennymack has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [13:13:00] Quit ekidd has left this server (Quit: ekidd).
[Monday 13 September 2010] [13:19:56] Quit omarkj has left this server (Quit: omarkj).
[Monday 13 September 2010] [13:27:35] Join AlexB has joined this channel (
alex@brasetvik.com).
[Monday 13 September 2010] [14:12:09] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [14:13:16] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [14:45:08] Quit ReinH has left this server (Ping timeout: 245 seconds).
[Monday 13 September 2010] [15:59:18] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [16:08:23] Quit rphillips has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [16:37:48] <ModusPwnens> Hi guys, I have a question on req/rep topology
[Monday 13 September 2010] [16:38:17] <ModusPwnens> Previously, I have been doing benchmarking with the subscriber/publisher toplogy, but I wanted to see what results I would get with req/rep
[Monday 13 September 2010] [16:38:52] <ModusPwnens> and I was wondering if there is anything else I need to do besides the obvious change of the socket types and adding in addition send/recv function calls to avoid blocking the code
[Monday 13 September 2010] [16:39:26] <ModusPwnens> because i noticed after I did those things, rather than sending a message of X bytes, it sends X messages of 1 byte
[Monday 13 September 2010] [16:41:05] <ModusPwnens> Actually, I lied. It seems to just send a lot of zero byte messages
[Monday 13 September 2010] [16:41:49] <cremes> ModusPwnens: pastie some code, because it should “just work”
[Monday 13 September 2010] [16:44:17] Quit gavinstark has left this server (Remote host closed the connection).
[Monday 13 September 2010] [16:51:29] <ModusPwnens> Actually, i figured out what it was. However, should rep/req have better or worse performance than sub/pub?
[Monday 13 September 2010] [17:07:24] <cremes> ModusPwnens: same perf but round-trip latency is higher (no such notion as round-trip latency with pub/sub)
[Monday 13 September 2010] [17:27:18] Quit plq has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [18:07:29] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [18:33:14] Quit ekidd has left this server (Quit: ekidd).
[Monday 13 September 2010] [18:46:37] Quit psino has left this server (Quit: psino).
[Monday 13 September 2010] [19:10:47] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [19:57:38] Quit rgl has left this server (Ping timeout: 264 seconds).
[Monday 13 September 2010] [20:24:01] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [20:39:05] Quit rphillips has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [20:50:48] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [20:54:32] Quit jhawk28 has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [20:59:33] Quit
jhawk28_ has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [21:05:12] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [21:10:53] Quit plq has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [21:15:35] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [21:42:44] Quit GeekGod has left this server (Quit: GeekGod).
[Monday 13 September 2010] [21:58:16] Quit grahamalot has left this server (Quit: grahamalot).
[Monday 13 September 2010] [22:00:53] Quit xla has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [22:17:53] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [22:30:44] Quit bgranger has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [22:45:29] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [23:39:45] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [00:52:17] Quit keffo has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [01:06:56] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [01:14:25] Quit psino has left this server (Quit: psino).
[Tuesday 14 September 2010] [01:55:59] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 14 September 2010] [01:56:36] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 14 September 2010] [01:58:14] Quit rgl has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [02:57:32] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [03:14:11] Quit jond has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [04:45:14] <pieterh> sustrik: ping?
[Tuesday 14 September 2010] [04:45:31] <sustrik> pieterh: pong
[Tuesday 14 September 2010] [04:45:42] <pieterh> is there any zero copy mechanism on reading messages?
[Tuesday 14 September 2010] [04:45:51] <pieterh> i.e. directly into application buffers
[Tuesday 14 September 2010] [04:46:45] <sustrik> no, you get the buffer from 0mq
[Tuesday 14 September 2010] [04:46:53] <sustrik> you can then store it as long as you wish
[Tuesday 14 September 2010] [04:47:12] <pieterh> ok, thx
[Tuesday 14 September 2010] [05:04:05] <ptrb> so you can
`zero-copy’ the buffer data beyond the lifetime of the zmq_msg_t?
[Tuesday 14 September 2010] [05:08:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [05:10:23] <sustrik> ptrb: no, but you can keep zmq_msg_t around as long as you wish
[Tuesday 14 September 2010] [05:10:34] <ptrb> roger.
[Tuesday 14 September 2010] [05:18:18] <pieterh> ptrb: I’ve added a problem solver to the guide
[Tuesday 14 September 2010] [05:20:46] <ptrb> cool
[Tuesday 14 September 2010] [05:21:00] <ptrb> the “Yes” path on “Are you using XREP sockets?” isn’t labeled :D
[Tuesday 14 September 2010] [05:21:08] <pieterh> thanks...!
[Tuesday 14 September 2010] [05:21:22] <ptrb> but that is definitely helpful
[Tuesday 14 September 2010] [05:21:40] <ptrb> i’m a little overwhelmed by the sheer number of different “guides” and “manuals” and whatnot on the docs page(s), though
[Tuesday 14 September 2010] [05:23:39] <pieterh> ptrb: yeah, standard process, lots of growth and then consolidation
[Tuesday 14 September 2010] [05:23:50] <pieterh> stick with www.zeromq.com for starters
[Tuesday 14 September 2010] [05:24:44] <pieterh> really there should only be two books, the guide and the reference manual
[Tuesday 14 September 2010] [05:24:55] <ptrb> sure
[Tuesday 14 September 2010] [05:25:16] <ptrb> there’s a solid case for a quick-start (~1h) tutorial as well, I think; maybe that’s like ch.1 of the guide
[Tuesday 14 September 2010] [05:25:39] <pieterh> yup
[Tuesday 14 September 2010] [05:25:55] <pieterh> plus lots of tutorials and articles from the rest of the community
[Tuesday 14 September 2010] [05:41:04] <keffo> pieterh, I have a concurrency issue that I cant really figure out!
[Tuesday 14 September 2010] [05:41:14] <pieterh> keffo: shoot
[Tuesday 14 September 2010] [05:41:31] <pieterh> have you checked the new problem solver?
[Tuesday 14 September 2010] [05:41:40] <keffo> lemme check
[Tuesday 14 September 2010] [05:42:57] <keffo> pieterh, Think client, double lru-queue(for outoing tasks, and for incoming results), then an additional ‘regular’ queue on each machine, which provides N worker processes..
[Tuesday 14 September 2010] [05:43:14] <keffo> pieterh, Theyre all sep. processes, no threading involved anywhere apart from zmq internals..
[Tuesday 14 September 2010] [05:43:22] <pieterh> ok
[Tuesday 14 September 2010] [05:43:53] <pieterh> what is the symptom you are seeing?
[Tuesday 14 September 2010] [05:44:29] <keffo> One worker, all is fine, I’ve done 100000 tasks back’n’forth with no problem, so the handling of data is sound.. Now if I do more than one worker processes, -sometimes- I get jumbled data, so it’s clearly a concurrency issue, but I cant really tell how or where
[Tuesday 14 September 2010] [05:45:12] <pieterh> jumbled data = one message has corrupted data or messages are not in order?
[Tuesday 14 September 2010] [05:45:34] <keffo> I guess both? =)
[Tuesday 14 September 2010] [05:45:43] <keffo> lemme trigger it and check the logs
[Tuesday 14 September 2010] [05:45:48] <pieterh> ok
[Tuesday 14 September 2010] [05:46:48] <keffo> there we go
[Tuesday 14 September 2010] [05:47:53] <keffo> so this scenario has multiple worker processes, but I send out batches or bursts of tasks of different granularity.. With 1 it always works, that is, send one task, wait for that task to complete.. The problem pops up when doing more than one, say 10
[Tuesday 14 September 2010] [05:48:00] <keffo> which is the case now
[Tuesday 14 September 2010] [05:48:31] <pieterh> keffo: you still need to explain what the “problem” actually is
[Tuesday 14 September 2010] [05:48:57] <pieterh> what is the difference between working and non-working in terms of what you see in the logs?
[Tuesday 14 September 2010] [05:49:43] <keffo> well, sometimes routing info is jumbled.. I expect uuid + nil + payload, but sometimes that uuid can get very weird
[Tuesday 14 September 2010] [05:50:01] <keffo> and likewise the payload itself can get messed up
[Tuesday 14 September 2010] [05:50:12] <pieterh> ok, so your message envelopes are wrong
[Tuesday 14 September 2010] [05:50:25] <keffo> They would be wrong with just one message too :)
[Tuesday 14 September 2010] [05:50:32] <keffo> or one worker proc.
[Tuesday 14 September 2010] [05:50:43] <pieterh> most probably an error in your code that creates/copies envelopers
[Tuesday 14 September 2010] [05:50:47] <pieterh>
*envelopes
[Tuesday 14 September 2010] [05:51:19] <pieterh> the LRU queue stuff is pretty delicate
[Tuesday 14 September 2010] [05:53:47] <keffo> I dont see the relevance to be honest, I can process 1M tasks properly if I wait for each message(or if I have only one worker processing them).. If there was an inherent problem in the envelope construction/usage, surely that would pop up there as well, but it has -never- happened in that scenario..
[Tuesday 14 September 2010] [05:54:02] Quit jsimmons has left this server (Ping timeout: 252 seconds).
[Tuesday 14 September 2010] [05:54:40] <pieterh> keffo: when problem solving you don’t care when it works
[Tuesday 14 September 2010] [05:54:52] <pieterh> if you can reproduce a problem, that’s the interesting thing
[Tuesday 14 September 2010] [05:55:06] <pieterh> things often work by accident...
[Tuesday 14 September 2010] [05:55:21] <pieterh> so your workers are REQ and you’re talking to them with an XREP?
[Tuesday 14 September 2010] [05:55:54] <keffo> yeah
[Tuesday 14 September 2010] [05:56:12] <pieterh> do you store the worker identity anywhere?
[Tuesday 14 September 2010] [05:56:23] <keffo> I can reproduce it, but with irregularities one has come to expect from multithreaded :)
[Tuesday 14 September 2010] [05:56:31] <keffo> yeah, all connections are known
[Tuesday 14 September 2010] [05:57:24] <pieterh> so the obvious thing to look at would be how you reconstruct the envelopes and where the worker addresses come from
[Tuesday 14 September 2010] [05:57:32] <pieterh> and that you’re not using the wrong data
[Tuesday 14 September 2010] [05:57:44] Quit omarkj has left this server (Read error: Connection reset by peer).
[Tuesday 14 September 2010] [05:58:34] <pieterh> you presumably have logic that is vulnerable to multiple workers sending back stuff at the same time
[Tuesday 14 September 2010] [05:59:04] <pieterh> maybe you’re not using SNDMORE properly in the worker when sending
[Tuesday 14 September 2010] [05:59:22] <pieterh> so messages arrive mixed together rather than atomically
[Tuesday 14 September 2010] [05:59:31] * pieterh has no idea if that is even possible...
[Tuesday 14 September 2010] [05:59:53] <keffo> right, I dont see that being possible either, as it properly routes across 3-4 queues?
[Tuesday 14 September 2010] [06:00:08] <pieterh> you have assertions on everything?
[Tuesday 14 September 2010] [06:00:20] <keffo> think so, yeah
[Tuesday 14 September 2010] [06:00:34] <pieterh> so every single frame is checked? e.g. RCVMORE where expected, empty frame where expected, etc.
[Tuesday 14 September 2010] [06:00:50] <keffo> yes
[Tuesday 14 September 2010] [06:01:05] <pieterh> the only think I can think of right now is that you can send what looks like a valid envelope but it won’t survive concurrency
[Tuesday 14 September 2010] [06:02:21] <keffo> What would cause multiple incoming multipart-messages to break symmetry with more than one sender, in a singlethreaded process?
[Tuesday 14 September 2010] [06:02:34] <keffo> that’s what I dont get :)
[Tuesday 14 September 2010] [06:02:47] <pieterh> well, if they are not actually multipart messages
[Tuesday 14 September 2010] [06:02:56] <pieterh> lets say you send the envelope (address, null, data)
[Tuesday 14 September 2010] [06:03:06] <pieterh> but you by mistake use normal sends on each part
[Tuesday 14 September 2010] [06:03:19] <keffo> yup
[Tuesday 14 September 2010] [06:03:21] <pieterh> it’ll be 3 messages, not 1 3-part message
[Tuesday 14 September 2010] [06:03:31] <pieterh> if you read that, it might look valid and work for routing
[Tuesday 14 September 2010] [06:03:40] <pieterh> but as soon as you have concurrency it’ll stop working
[Tuesday 14 September 2010] [06:03:56] <pieterh> like i say, i’ve never tried that, but will make a test now, after coffee
[Tuesday 14 September 2010] [06:04:12] <pieterh> i’d expect 0MQ to reject such messages...
[Tuesday 14 September 2010] [06:04:29] <keffo> hmm.. I’ll go over the code again! :)
[Tuesday 14 September 2010] [06:04:32] <pieterh> but from REQ to XREP it can’t IMO
[Tuesday 14 September 2010] [06:06:40] <keffo> cant reject?
[Tuesday 14 September 2010] [06:07:04] <sustrik> keffo: can you check whether the garbled identity you see is something that may actually your application message or whether it’s just real junk
[Tuesday 14 September 2010] [06:07:05] <sustrik> ?
[Tuesday 14 September 2010] [06:09:20] <keffo> It’s not real junk.. Sometimes for example I get route uuid that appears to be a mix of multiple uuid’s
[Tuesday 14 September 2010] [06:09:41] <keffo> clearly something breaks symmetry somehow
[Tuesday 14 September 2010] [06:09:50] <keffo> I’ll lok into the route logic again though
[Tuesday 14 September 2010] [06:11:56] <sustrik> “mix of multiple uuid’s” = junk IMO
[Tuesday 14 September 2010] [06:12:08] <sustrik> ok, a sanity check:
[Tuesday 14 September 2010] [06:12:20] <sustrik> do you use each socket exclusively from a single thread?
[Tuesday 14 September 2010] [06:12:44] <keffo> yeah
[Tuesday 14 September 2010] [06:12:54] <sustrik> what language are you using?
[Tuesday 14 September 2010] [06:13:02] <keffo> c++ and lua
[Tuesday 14 September 2010] [06:13:10] <keffo> but homebrew bindings
[Tuesday 14 September 2010] [06:13:42] <sustrik> any chance to test a pure c++ setup?
[Tuesday 14 September 2010] [06:13:55] <keffo> hehe. noo :)
[Tuesday 14 September 2010] [06:14:35] <sustrik> hm, lua binding is a project i know nothing about
[Tuesday 14 September 2010] [06:15:02] <sustrik> the bug can be there, as no such problem was yet reported for any other language
[Tuesday 14 September 2010] [06:15:30] <sustrik> ok, any chance of using anything other than lua?
[Tuesday 14 September 2010] [06:19:11] <keffo> Would take forever to rewrite
[Tuesday 14 September 2010] [06:19:20] <sustrik> ok
[Tuesday 14 September 2010] [06:19:32] <sustrik> how does the setup look like?
[Tuesday 14 September 2010] [06:19:40] <sustrik> which part is c++ and which is lua?
[Tuesday 14 September 2010] [06:20:03] <keffo> all zmq stuff is c, rest is lua.. Ie, connect/bind/send/recv(and their more-counterparts)
[Tuesday 14 September 2010] [06:20:51] <sustrik> you are mixing C code and lua code to use the same 0MQ socket?
[Tuesday 14 September 2010] [06:21:01] * sustrik doesn’t follow
[Tuesday 14 September 2010] [06:21:22] <keffo> Um no, I created bindings for sockets, so it’s the same thing
[Tuesday 14 September 2010] [06:21:50] <sustrik> hm, are you saying that 0MQ is written in C while your apps are in lua?
[Tuesday 14 September 2010] [06:22:29] <keffo> I’ve extended an application with lua yeah..
[Tuesday 14 September 2010] [06:22:34] <keffo> lua or not is besides the point really
[Tuesday 14 September 2010] [06:22:47] <keffo> All zmq related code is in C++
[Tuesday 14 September 2010] [06:23:24] <sustrik> aha, so you are _not_ using the lua binding
[Tuesday 14 September 2010] [06:23:26] <sustrik> right?
[Tuesday 14 September 2010] [06:23:44] <keffo> I am, but not the ones from the site :)
[Tuesday 14 September 2010] [06:23:50] <sustrik> your own
[Tuesday 14 September 2010] [06:23:55] <keffo> yup
[Tuesday 14 September 2010] [06:24:14] <keffo> just an abstraction layer that’s easier to work with on the lua side
[Tuesday 14 September 2010] [06:24:20] <sustrik> ack
[Tuesday 14 September 2010] [06:24:29] <sustrik> what’s the lua’s threading model?
[Tuesday 14 September 2010] [06:24:40] <keffo> inherently single threaded, but it has coroutines
[Tuesday 14 September 2010] [06:25:38] <sustrik> ok, so there’s only one OS thread for each lua process, right?
[Tuesday 14 September 2010] [06:25:42] <keffo> yes
[Tuesday 14 September 2010] [06:26:22] <sustrik> no such think as garbage collection thread or similar?
[Tuesday 14 September 2010] [06:26:25] <sustrik> thing*
[Tuesday 14 September 2010] [06:26:26] <keffo> client, loadbalancer, computenode and workers are all single processes (a computenode is simply a c++ queue dev. ‘owning’ the worker processes)
[Tuesday 14 September 2010] [06:26:52] <keffo> No, not in another thread no.. Lua does incremental gc but in the same thread
[Tuesday 14 September 2010] [06:27:11] <sustrik> ok
[Tuesday 14 September 2010] [06:27:37] <sustrik> then it looks like the problem has to be in 0MQ itself
[Tuesday 14 September 2010] [06:27:46] <sustrik> it should be reproducible in C or C++
[Tuesday 14 September 2010] [06:28:15] <sustrik> can you write a simpe C/C++ test that simulates what you are doing in your app?
[Tuesday 14 September 2010] [06:28:35] <sustrik> messaging-wise i mean, not the business logic
[Tuesday 14 September 2010] [06:28:47] <keffo> hmm.. that would take a while I think..
[Tuesday 14 September 2010] [06:29:07] <keffo> I do think I’m to blame, not zmq, I just need a better way to debug it probably :)
[Tuesday 14 September 2010] [06:30:01] <keffo> It behaves as threaded concurrency issue, but being entirely single process based..
[Tuesday 14 September 2010] [06:30:37] <sustrik> there’s an I/O thread inside 0MQ so it can be a race
[Tuesday 14 September 2010] [06:31:00] <sustrik> however, i can’t help unless i am able to reproduce it
[Tuesday 14 September 2010] [06:31:20] <keffo> Well I’d be half way safe if I could reproduce it too, but it’s quite rare :)
[Tuesday 14 September 2010] [06:31:37] <sustrik> of course, it’s still possible that you are overwriting memory in your app or something like that
[Tuesday 14 September 2010] [06:31:54] <sustrik> have you tried to valgring your binding?
[Tuesday 14 September 2010] [06:31:58] <sustrik> valgrind
[Tuesday 14 September 2010] [06:33:17] <keffo> no valgrind in win as far as I know.. Boundschecker perhaps, but I doubt that’s the problem too
[Tuesday 14 September 2010] [06:33:41] <keffo> Again, in a single-worker scenario, everything is flawless regardless of amount and load etc
[Tuesday 14 September 2010] [06:34:00] <keffo> (and none of the processes every leak anything, stays solid around 2mb mem)
[Tuesday 14 September 2010] [06:34:39] <sustrik> then it looks like a problem in the central dispatcher node
[Tuesday 14 September 2010] [06:34:50] <keffo> yeah
[Tuesday 14 September 2010] [06:34:57] <keffo> that’s the most complicated bit too
[Tuesday 14 September 2010] [06:35:09] <keffo> double lru queue device basically
[Tuesday 14 September 2010] [06:36:01] <sustrik> well, unless you are able to provide something to reproduce the problem, you are on your own here
[Tuesday 14 September 2010] [06:36:08] <keffo> I know :)
[Tuesday 14 September 2010] [06:37:02] <keffo> more logging I guess. It would be nice to have all participants publish all the data they process, and collect that in a sub app, and then be able to “play back” everything
[Tuesday 14 September 2010] [06:37:30] <ptrb> that could easily become a project as complex as zmq itself
[Tuesday 14 September 2010] [06:37:43] <ptrb> entire companies are built on the concept of capturing/replaying network traffic
[Tuesday 14 September 2010] [06:37:52] <pieterh> keffo: if you can send me your code I’ll see if I can spot anything
[Tuesday 14 September 2010] [06:38:03] <pieterh> i need the parts that read and write worker envelopes, to start with
[Tuesday 14 September 2010] [06:38:12] <keffo> sure hold on
[Tuesday 14 September 2010] [06:38:17] <keffo> wheres the pastebin?
[Tuesday 14 September 2010] [06:38:29] <pieterh> gist.github.com works well
[Tuesday 14 September 2010] [06:39:04] <keffo> But again, if the envelope code is flawed, it would be equally flawed for one worker?
[Tuesday 14 September 2010] [06:40:27] <pieterh> keffo: i explained how it could be flawed but still work for one worker
[Tuesday 14 September 2010] [06:42:22] <pieterh> keffo: i need the actual detail of sendpath, I think
[Tuesday 14 September 2010] [06:42:27] <pieterh>
*sendmore, sorry
[Tuesday 14 September 2010] [06:42:45] <keffo> That is in C, hold on
[Tuesday 14 September 2010] [06:44:52] <keffo> send() is obviously quite similar except for the flag :)
[Tuesday 14 September 2010] [06:45:56] <pieterh> keffo: it looks fine
[Tuesday 14 September 2010] [06:46:15] <pieterh> you’re sure that pairs() gives you stuff in the right order?
[Tuesday 14 September 2010] [06:46:38] <pieterh> so you have only one choice left, sadly
[Tuesday 14 September 2010] [06:46:55] <keffo> rewrite, and hope it goes away? =)
[Tuesday 14 September 2010] [06:47:05] <keffo> naa, there’s some stupid silly thing somewhere
[Tuesday 14 September 2010] [06:47:13] <pieterh> well, everything goes through these functions
[Tuesday 14 September 2010] [06:47:20] <pieterh> so put in printf statements or whatever
[Tuesday 14 September 2010] [06:47:27] <pieterh> and see when it actually goes wrong
[Tuesday 14 September 2010] [06:47:46] <keffo> Yeah, need to find a pattern..
[Tuesday 14 September 2010] [06:47:50] <pieterh> pieter’s rules #5 and #23 of debugging
[Tuesday 14 September 2010] [06:48:03] <pieterh> (a) it’s never actually a race condition
[Tuesday 14 September 2010] [06:48:14] <pieterh> (b) you can find any bug using printfs
[Tuesday 14 September 2010] [06:48:23] <keffo> except the ones in printf!
[Tuesday 14 September 2010] [06:48:37] <pieterh> that’s why C offers puts as well!
[Tuesday 14 September 2010] [06:49:39] <keffo> One thing I could do is rewrite the loadbalancer stuff in pure C
[Tuesday 14 September 2010] [06:50:11] <pieterh> keffo: there’s no point rewriting unless you have found the cause of the problem
[Tuesday 14 September 2010] [06:50:31] <ptrb> why are your rule #5 and #23 labelled (a) and (b)??
[Tuesday 14 September 2010] [06:50:31] <keffo> indeed
[Tuesday 14 September 2010] [06:51:04] <pieterh> ptrb: lol
[Tuesday 14 September 2010] [06:51:04] <keffo> pieterh, That code should be rewritten though, for performance, but I agree :)
[Tuesday 14 September 2010] [07:33:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [07:37:02] <BooTheHamster> What mean error “operation cannot be accomplished in current state” in zmq_send() call?
[Tuesday 14 September 2010] [07:38:51] <ptrb> hey i got that yesterday! :D
[Tuesday 14 September 2010] [07:39:00] <ptrb> basically means you’re violating some constraint of the socket
[Tuesday 14 September 2010] [07:39:08] <ptrb> is it ZMQ_REQ or _REP?
[Tuesday 14 September 2010] [07:39:15] <BooTheHamster> ZMQ_REQ
[Tuesday 14 September 2010] [07:39:37] <ptrb> you must call zmq_send, then zmq_recv, before you can call zmq_send again
[Tuesday 14 September 2010] [07:39:39] <BooTheHamster> I’t a client app but run withiut server
[Tuesday 14 September 2010] [07:39:39] <ptrb> are you doing that?
[Tuesday 14 September 2010] [07:39:46] <BooTheHamster> yes
[Tuesday 14 September 2010] [07:39:55] <BooTheHamster> first I call zmq_send
[Tuesday 14 September 2010] [07:40:05] <BooTheHamster> and then zmq_recv
[Tuesday 14 September 2010] [07:40:11] <ptrb> on the same socket
[Tuesday 14 September 2010] [07:40:19] <BooTheHamster> m_MsgSock->recv(&msg_rcv, ZMQ_NOBLOCK)
[Tuesday 14 September 2010] [07:40:21] <BooTheHamster> yes
[Tuesday 14 September 2010] [07:40:33] <ptrb> noblock means you may not actually have recv’d anything
[Tuesday 14 September 2010] [07:40:46] <ptrb> you have to actually recv something before you can send again (I believe–maybe someone can confirm)
[Tuesday 14 September 2010] [07:41:21] <BooTheHamster> but i want non blocking recv
[Tuesday 14 September 2010] [07:41:36] <ptrb> that’s fine, you just can’t send again until your non-blocking recv actually gets something
[Tuesday 14 September 2010] [07:41:57] <ptrb> if you want to do multiple sends without getting data back, use XREQ
[Tuesday 14 September 2010] [07:43:41] <BooTheHamster> Thank you ptrb, XREQ helps.
[Tuesday 14 September 2010] [07:44:29] <BooTheHamster> A little one question ... on server side i must use XREP or REP socket?
[Tuesday 14 September 2010] [07:45:36] <ptrb> just depends on what behavior you can tolerate there... both will work, but REP requires you to recv, send, recv, send actual data
[Tuesday 14 September 2010] [07:46:08] <ptrb>
`man zmq_socket’ for more details
[Tuesday 14 September 2010] [07:50:49] <BooTheHamster> thanks
[Tuesday 14 September 2010] [07:52:34] Part BooTheHamster has left this channel.
[Tuesday 14 September 2010] [07:52:40] <ptrb> i helped!! :D
[Tuesday 14 September 2010] [07:52:44] <ptrb> maybe
[Tuesday 14 September 2010] [07:55:42] Quit psino has left this server (Quit: psino).
[Tuesday 14 September 2010] [08:04:35] Part icy has left this channel.
[Tuesday 14 September 2010] [08:25:25] <cremes> ptrb: you’re an expert now! ;)
[Tuesday 14 September 2010] [08:46:18] Quit jsimmons has left this server (Quit: Leaving).
[Tuesday 14 September 2010] [08:48:39] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Tuesday 14 September 2010] [08:51:23] Quit xla has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [09:02:00] <pieterh> cremes: I moved your xrep-xrep recipe page into the tutorials section
[Tuesday 14 September 2010] [09:07:28] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 14 September 2010] [09:27:17] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [09:47:35] <Tasser> why use rbzmq <=> ffi-rzmq?
[Tuesday 14 September 2010] [09:54:46] <cremes> Tasser: they have slightly different APIs; the ffi-rzmq more closely tracks the C api whereas rbzmq uses strings for everything
[Tuesday 14 September 2010] [09:55:02] <cremes> also, ffi-rzmq is more compatible with ruby runtimes (jruby, rubinius, mri)
[Tuesday 14 September 2010] [09:55:08] <Tasser> strings sounds like overuse of core classes
[Tuesday 14 September 2010] [09:55:37] <cremes> well, most people will probably convert their data to string format for send/recv under ruby because it’s convenient
[Tuesday 14 September 2010] [09:55:54] <Tasser> that’s what JSON is for :-)
[Tuesday 14 September 2010] [09:56:10] <cremes> but since ffi-rzmq exposes a zmq_msg_t-like structure you can pass around C structs without doing a lot of extra copying
[Tuesday 14 September 2010] [09:56:15] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [09:56:15] <Tasser> but I suspect you can’t hardwire zmq against json
[Tuesday 14 September 2010] [09:56:18] <cremes> that might be more efficient
[Tuesday 14 September 2010] [09:56:40] <cremes> what do you mean by hardware? as far as i know, 0mq (and zmq) don’t care what you pass on the wire
[Tuesday 14 September 2010] [09:56:47] <cremes> s/hardware/hardwire/
[Tuesday 14 September 2010] [09:57:07] <Tasser> well, for filtering in subscribe
[Tuesday 14 September 2010] [09:57:28] <cremes> ah, subscribe filters at the byte-level
[Tuesday 14 September 2010] [09:57:55] <cremes> no need to encode the subscription topic unless you want to decode it on each subscriber too
[Tuesday 14 September 2010] [09:59:20] <Tasser> hmm
[Tuesday 14 September 2010] [10:00:50] <cremes> hmm sometimes means “i don’t know what you are talking about” :)
[Tuesday 14 September 2010] [10:10:21] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [10:51:56] Quit bgranger has left this server (Client Quit).
[Tuesday 14 September 2010] [10:58:00] <Tasser> cremes, well, so you could filter based on JSON, but that would make it bloat
[Tuesday 14 September 2010] [10:58:28] Join jond has joined this channel (
~jon@87.74.16.24).
[Tuesday 14 September 2010] [10:59:41] <cremes> Tasser: my point is that 0mq matches topics by comparing bytes
[Tuesday 14 September 2010] [10:59:48] <cremes> so you want it to fail as soon as possible
[Tuesday 14 September 2010] [11:00:05] <cremes> using a json-encoded topic means that more bytes would need to be compared to find mismatches
[Tuesday 14 September 2010] [11:00:13] <cremes> so subscription filtering would be slower
[Tuesday 14 September 2010] [11:00:23] <cremes> for example...
[Tuesday 14 September 2010] [11:00:51] <cremes> sock.setsockopt ZMQ::SUBSCRIBE, ‘topic.string’
[Tuesday 14 September 2010] [11:00:53] <cremes> versus
[Tuesday 14 September 2010] [11:01:25] <cremes> sock.setsockopt(ZMQ::SUBSCRIBE, JSON.encode({‘topic’ => ‘topic.string’})
[Tuesday 14 September 2010] [11:01:36] <cremes> honestly, i’m not even sure if that second example would ever work
[Tuesday 14 September 2010] [11:01:53] <cremes> since the json encoder could put things in different order while still producing valid json
[Tuesday 14 September 2010] [11:02:10] <cremes> just use a string for the topic and put your encoded data into a second message part
[Tuesday 14 September 2010] [11:02:48] <Tasser> yeah, probably the sane idea of using it.
[Tuesday 14 September 2010] [11:50:19] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Tuesday 14 September 2010] [11:53:31] Quit zoobab has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [12:05:40] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [12:11:28] Quit jdknezek has left this server (Quit: Page closed).
[Tuesday 14 September 2010] [12:36:34] Quit
keffo_ has left this server (Ping timeout: 265 seconds).
[Tuesday 14 September 2010] [13:00:39] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Tuesday 14 September 2010] [13:01:00] <kleppari> hello
[Tuesday 14 September 2010] [13:02:38] <ModusPwnens> Im not sure, but if you do register, let me know if you are successful. I was trying to the other day and it wouldn’t work =/
[Tuesday 14 September 2010] [13:03:01] <pieterh> kleppari: isn’t is showing? rats... :-( thanks for pointing it out
[Tuesday 14 September 2010] [13:03:08] <guido_g> f*ck, this is wikidot crap
[Tuesday 14 September 2010] [13:03:14] <guido_g> sorry
[Tuesday 14 September 2010] [13:03:32] <pieterh> gudio_g: default permissions on cloned site are private, I forgot
[Tuesday 14 September 2010] [13:03:54] <guido_g> and you need to allow cross-domain cookies
[Tuesday 14 September 2010] [13:03:57] <pieterh> happily it’s a 10-second fix
[Tuesday 14 September 2010] [13:04:09] <guido_g> wikidot sucks in this regard
[Tuesday 14 September 2010] [13:04:10] <pieterh> guido_g: you need to not disallow the dratted things
[Tuesday 14 September 2010] [13:04:21] <pieterh> it’s for good security reasons, unfortunately
[Tuesday 14 September 2010] [13:04:32] <kleppari> registered with wikidots, worked after that. I think it’s a major fail to require it though :)
[Tuesday 14 September 2010] [13:04:43] <pieterh> kleppari: not required, it was my fault
[Tuesday 14 September 2010] [13:04:44] <kleppari> pieterh: np, thanks for the guide :)
[Tuesday 14 September 2010] [13:04:49] <guido_g> pieterh: cross-donmain cookies are for security?
[Tuesday 14 September 2010] [13:04:50] <pieterh> it should work for anonymous users now
[Tuesday 14 September 2010] [13:05:08] <pieterh> guido_g: wikidot splits off insecure stuff (file uploads) to a separate domain
[Tuesday 14 September 2010] [13:05:08] <guido_g> nack
[Tuesday 14 September 2010] [13:05:30] <guido_g> pieterh: it doesen’t work
[Tuesday 14 September 2010] [13:05:30] <pieterh> feel free to debate with the wikidot devs, they are pretty sure of their skills in this respect
[Tuesday 14 September 2010] [13:05:46] <pieterh> what doesn’t work?
[Tuesday 14 September 2010] [13:05:48] <kleppari> pieterh: works now, thanks.
[Tuesday 14 September 2010] [13:05:52] <pieterh> ?
[Tuesday 14 September 2010] [13:06:03] <kleppari> i.e., anonymous users on zguide.zeromq.org
[Tuesday 14 September 2010] [13:06:11] <kleppari> oh, no, sorry
[Tuesday 14 September 2010] [13:06:14] <guido_g> w/ the damn cookies
[Tuesday 14 September 2010] [13:06:51] <pieterh> guido_g: anything different than zeromq.org?
[Tuesday 14 September 2010] [13:07:03] <pieterh> there’s nothing special about this site afaics
[Tuesday 14 September 2010] [13:07:06] <Samy> sustrik, ping?
[Tuesday 14 September 2010] [13:07:21] <guido_g> i allowed the cookies for zeromq.org and the wikidot server
[Tuesday 14 September 2010] [13:07:49] <guido_g> but i’m not going to do that for every new site
[Tuesday 14 September 2010] [13:07:55] <pieterh> ah, well, allow them for
*.zeromq.org and
*.zeromq.com :-)
[Tuesday 14 September 2010] [13:08:08] <pieterh> why do you disallow them?
[Tuesday 14 September 2010] [13:08:18] <guido_g> i do not allow cookies generally
[Tuesday 14 September 2010] [13:08:26] <pieterh> well...
[Tuesday 14 September 2010] [13:08:48] <pieterh> kleppari: thanks, I’ll try it...
[Tuesday 14 September 2010] [13:08:51] * pieterh recalls people saying “I don’t use JavaScript”
[Tuesday 14 September 2010] [13:09:14] <pieterh> unfortunately interactive websites kind of need stuff like that
[Tuesday 14 September 2010] [13:09:43] <guido_g> viewing a page is not interactive
[Tuesday 14 September 2010] [13:10:08] <kleppari> sorry, screwed up file ownership. the screenshot of unauthenticated zguide.zeromq.org should be ok now.
[Tuesday 14 September 2010] [13:10:41] <pieterh> kleppari, hang on, 30 seconds... I’ll fix it, thanks for the screenshot
[Tuesday 14 September 2010] [13:11:10] <pieterh> can you try to reload the site please?
[Tuesday 14 September 2010] [13:11:13] <pieterh> ctrl-R
[Tuesday 14 September 2010] [13:11:23] <kleppari> like a glove
[Tuesday 14 September 2010] [13:11:25] <kleppari> thanks :)
[Tuesday 14 September 2010] [13:11:39] <pieterh> i didn’t change anything, it was already working... :-)
[Tuesday 14 September 2010] [13:11:50] <pieterh> since 19:03 CET
[Tuesday 14 September 2010] [13:12:10] <pieterh> ok, kleppari I appreciate the heads up, kind of stupid to have a Guide no-one can read... :-)
[Tuesday 14 September 2010] [13:12:26] <kleppari> screenshot snapped at 17:06 GMT. Weird.
[Tuesday 14 September 2010] [13:12:32] <kleppari> you’re welcome
[Tuesday 14 September 2010] [13:12:56] <pieterh> guido_g, yeah, because Sustrik pointed out that parts of it were bogus
[Tuesday 14 September 2010] [13:13:00] <pieterh> i’m rewriting it
[Tuesday 14 September 2010] [13:13:05] <pieterh> as soon as i finish my pasta and wine
[Tuesday 14 September 2010] [13:13:09] <guido_g> and why this intermediate page when i click in “The Guide” from zeromq.org?
[Tuesday 14 September 2010] [13:13:37] <pieterh> you’d rather leap to chapter 1 right away?
[Tuesday 14 September 2010] [13:13:48] <guido_g> yes
[Tuesday 14 September 2010] [13:13:55] <pieterh> i mean when you get a book, you see the cover first...
[Tuesday 14 September 2010] [13:14:07] <pieterh> but ok, can be done...
[Tuesday 14 September 2010] [13:14:08] <guido_g> the link is named “The Guide”
[Tuesday 14 September 2010] [13:14:24] <pieterh> Yeah, not The Guide Chapter 1 :-)
[Tuesday 14 September 2010] [13:14:36] <guido_g> not “promotional intermediate page w/o content”
[Tuesday 14 September 2010] [13:14:54] <pieterh> hey, don’t mock the promotional pages...
[Tuesday 14 September 2010] [13:15:22] <guido_g> only if you reduce the size of the MQ logo drastically
[Tuesday 14 September 2010] [13:15:25] <pieterh> delayed gratification...
[Tuesday 14 September 2010] [13:15:40] <pieterh> OMG don’t touch the logo, it’s sacred...
[Tuesday 14 September 2010] [13:15:44] <guido_g> btw, too much red makes aggressive :)
[Tuesday 14 September 2010] [13:15:53] <pieterh> ok, the link from the sidebad should now go straight to the first chapter
[Tuesday 14 September 2010] [13:16:20] <guido_g> nope
[Tuesday 14 September 2010] [13:16:33] * pieterh is on a lousy 2G modem that takes ages
[Tuesday 14 September 2010] [13:16:55] <guido_g> ok, i’ll wait
[Tuesday 14 September 2010] [13:17:20] <pieterh> guido_g, when you say “nope” you mean as in “oops, I didn’t reload it and HTTP doesn’t yet have a autoupdate function”...?
[Tuesday 14 September 2010] [13:17:57] <guido_g> pieterh: no, it means “i pressed relaod like maniac but the thing didn’t work”
[Tuesday 14 September 2010] [13:18:09] <pieterh> works for me...
[Tuesday 14 September 2010] [13:18:22] <pieterh> go back to www.zeromq.org
[Tuesday 14 September 2010] [13:18:46] Quit speedy1 has left this server (Quit: Leaving).
[Tuesday 14 September 2010] [13:18:47] <pieterh> are there more links? I fixed the one in the sidebar
[Tuesday 14 September 2010] [13:18:51] <pieterh> what one are you clicking on?
[Tuesday 14 September 2010] [13:18:54] <guido_g> i am on zeromq.org and open the link in another tab
[Tuesday 14 September 2010] [13:19:04] <guido_g> after reloading the page
[Tuesday 14 September 2010] [13:19:49] <pieterh> shrug...it’s correct... you are reloading zeromq.org?
[Tuesday 14 September 2010] [13:19:57] <guido_g> yes
[Tuesday 14 September 2010] [13:20:16] <guido_g> hmmm... cdn thing maybe
[Tuesday 14 September 2010] [13:20:25] <pieterh> did you sacrifice a chicken to the gnomes of the Internet?
[Tuesday 14 September 2010] [13:20:53] <guido_g> ahhh... i constantly miss this one (eating the chicken myself)
[Tuesday 14 September 2010] [13:21:07] <pieterh> might be some caching somewhere your side of things
[Tuesday 14 September 2010] [13:21:19] <pieterh> it’s correct, anyhow, just tested again
[Tuesday 14 September 2010] [13:21:31] <guido_g> ahhh... cdn got the update
[Tuesday 14 September 2010] [13:21:35] <kleppari> you might also be clicking different links?
[Tuesday 14 September 2010] [13:21:40] <guido_g> no
[Tuesday 14 September 2010] [13:22:09] <kleppari> the sidebar link points to chapter one, the other guide links point to the ‘promotional’ page
[Tuesday 14 September 2010] [13:22:27] * pieterh looks for the other guide links
[Tuesday 14 September 2010] [13:22:29] <guido_g> so remember, it might take approx. 5 minutes til changes become vsisible
[Tuesday 14 September 2010] [13:22:41] <kleppari> two of them, one under most popular, other in recent site changes
[Tuesday 14 September 2010] [13:22:51] <pieterh> ah, kleppari, right
[Tuesday 14 September 2010] [13:23:19] <kleppari> but.. After a little while I like the promotional page
[Tuesday 14 September 2010] [13:23:28] <pieterh> well, indeed, a book deserves a cover
[Tuesday 14 September 2010] [13:23:40] <pieterh> bright red
[Tuesday 14 September 2010] [13:24:17] <pieterh> anyhow, this is always a work in progress, i like to see how people respond over time before making more changes
[Tuesday 14 September 2010] [13:24:33] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 14 September 2010] [13:24:48] <kleppari> I didn’t get it at first though, started at zeromq.com -> hit read the manual -> clicked ‘MQ - The guide’ -> got to zguide.zeromq.org
[Tuesday 14 September 2010] [13:24:51] <guido_g> pieterh: you’re on a laptop?
[Tuesday 14 September 2010] [13:25:01] <kleppari> didn’t catch the differences between zguide.zeromq.org and zeromq.com right away
[Tuesday 14 September 2010] [13:25:13] <pieterh> guido_g, on a laptop with a 2G phone modem, eating pasta in a hotel
[Tuesday 14 September 2010] [13:25:26] <guido_g> it’S more about the display size
[Tuesday 14 September 2010] [13:25:32] <kleppari> sounds like a crap hotel :P
[Tuesday 14 September 2010] [13:25:53] <pieterh> kleppari, it’s a pretty amazing hotel actually, an old Soviet relic
[Tuesday 14 September 2010] [13:26:04] <pieterh> guido_g, yes, laptop display
[Tuesday 14 September 2010] [13:26:38] <guido_g> pieterh: because on my 13” display the logo takes up apporx. 1/3 of the vertical space
[Tuesday 14 September 2010] [13:26:51] <guido_g> which is way to much, imho
[Tuesday 14 September 2010] [13:27:08] <guido_g> worse if i zoom in for better readability
[Tuesday 14 September 2010] [13:27:18] <pieterh> guido_g, yes, on mine too but it doesn’t matter on the cover page, and hardly matters on the content pages since you need to scroll anyhow
[Tuesday 14 September 2010] [13:27:33] <kleppari> looks good on 1280x1024
[Tuesday 14 September 2010] [13:27:42] <pieterh> we tried using a smaller logo some time ago, people felt it wasn’t nice
[Tuesday 14 September 2010] [13:28:01] <kleppari> pieterh: and no rj45 jack? URL? :)
[Tuesday 14 September 2010] [13:28:20] <guido_g> ok, i give up
[Tuesday 14 September 2010] [13:28:27] <pieterh> kleppari, URL? no rj45, no, wifi but it’s too slow
[Tuesday 14 September 2010] [13:28:48] <kleppari> url of the hotel website? or is it too much of a relic?
[Tuesday 14 September 2010] [13:28:49] <pieterh> guido_g, do start a discussion on look and feel, but this isn’t the place IMO
[Tuesday 14 September 2010] [13:29:23] <pieterh> guido_g, the original look was designed by a proper artist, and anything that replaces it would have to be done carefully
[Tuesday 14 September 2010] [13:29:32] <pieterh> kleppari, it’s the Hotel Kyjev in Bratislava
[Tuesday 14 September 2010] [13:30:38] <guido_g> pieterh: i didn’t complain about the logo, only it’s size, but nevermind i’ll find a solution
[Tuesday 14 September 2010] [13:30:57] <pieterh> guido_g, I do agree the logo is HUGE but my previous attempts to downsize it failed
[Tuesday 14 September 2010] [13:31:23] <guido_g> ok
[Tuesday 14 September 2010] [13:32:11] <pieterh> guido_g, what might work is a lighter theme for sites other than the Start Here site
[Tuesday 14 September 2010] [13:32:23] <pieterh> now that we have more sites, we can fine-tune them
[Tuesday 14 September 2010] [13:38:04] <guido_g> except for the “promo” pages the focus should be on readability
[Tuesday 14 September 2010] [13:42:05] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [13:53:08] Quit
pieterh_ has left this server (Read error: No route to host).
[Tuesday 14 September 2010] [14:52:29] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [14:54:31] Quit xla has left this server (Ping timeout: 255 seconds).
[Tuesday 14 September 2010] [15:29:26] Quit
pieterh__ has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [16:18:53] Quit benoitc has left this server (Ping timeout: 245 seconds).
*** on Tue Sep 14 16:37:22 2010
[Tuesday 14 September 2010] [16:37:22] Topic The channel topic is “Welcome!”.
[Tuesday 14 September 2010] [16:37:28] Mode Channel modes: no messages from outside, topic protection
[Tuesday 14 September 2010] [16:37:28] Created This channel was created on 2010-02-05 10:44.
[Tuesday 14 September 2010] [16:44:44] Part bobs has left this channel.
[Tuesday 14 September 2010] [17:10:43] Quit rphillips has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [18:06:56] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [18:43:28] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [18:57:59] Quit plq has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [19:40:23] Quit bgranger has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [19:44:27] Quit
bgranger_ has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [20:42:43] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [21:50:01] <kenkeiter> Okay, Rubyists with zmq experience – question!
[Tuesday 14 September 2010] [21:55:01] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [21:55:09] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [22:20:34] Quit AlexB has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [22:20:38] Join AlexB has joined this channel (
alex@brasetvik.com).
[Tuesday 14 September 2010] [22:23:00] <
Xin_> test
[Tuesday 14 September 2010] [22:25:21] <
Xin_> Does it make sense to add UDP support to 0mq?
[Tuesday 14 September 2010] [22:27:20] Quit
Xin_ has left this server (Quit: Page closed).
[Tuesday 14 September 2010] [22:47:48] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Tuesday 14 September 2010] [22:56:32] Quit kenkeiter has left this server (Quit: kenkeiter).
[Tuesday 14 September 2010] [22:57:34] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [23:27:41] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [23:43:31] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [00:03:41] Quit rgl has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [00:54:36] Join kenkeiter has joined this channel (
~kenkeiter@2002:4c73:685:1234:223:6cff:fe98:be3b).
[Wednesday 15 September 2010] [00:54:47] Quit kenkeiter has left this server (Client Quit).
[Wednesday 15 September 2010] [01:16:01] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 15 September 2010] [01:17:36] Quit guido_g has left this server (Client Quit).
[Wednesday 15 September 2010] [01:32:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [02:23:26] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [02:24:37] Quit bgranger has left this server (Client Quit).
[Wednesday 15 September 2010] [02:26:58] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 15 September 2010] [02:31:02] Quit keffo has left this server (Ping timeout: 272 seconds).
[Wednesday 15 September 2010] [02:47:24] Quit carrus85 has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [03:02:03] Quit kleppari has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [03:04:21] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [03:10:47] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Wednesday 15 September 2010] [03:15:41] Quit
AndrewBC_ has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [03:22:45] Quit plq has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [03:32:39] Quit rgl has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [04:06:06] <sustrik> Samy: pong
[Wednesday 15 September 2010] [04:06:38] <Zao> Doesn’t PGM stuff travel on UDP already?
[Wednesday 15 September 2010] [04:06:48] * Zao notes the six hour gap in time.
[Wednesday 15 September 2010] [04:09:05] <sustrik> Zao: PGM travels on IP
[Wednesday 15 September 2010] [04:09:22] <sustrik> epgm is a convenience hack that travels on UDP instead of IP
[Wednesday 15 September 2010] [04:09:32] <Zao> Oh, right.
[Wednesday 15 September 2010] [04:09:43] <Zao> Never used any of them, so I went with faint memories of manpages past.
[Wednesday 15 September 2010] [04:11:43] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Wednesday 15 September 2010] [04:53:30] Quit manveru has left this server (Quit: WeeChat 0.3.2).
[Wednesday 15 September 2010] [05:15:54] Join omarkj has joined this channel (
~omarkj@81.15.34.139).
[Wednesday 15 September 2010] [05:18:46] <omarkj> kleppari: Morning.
[Wednesday 15 September 2010] [05:28:58] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [05:29:15] <mrm2m> Hey ho!
[Wednesday 15 September 2010] [05:30:25] <mrm2m> Is there a lightweight introduction, how zmq works internally?
[Wednesday 15 September 2010] [05:33:01] <pieterh> mrm2m: there is documentation of the source code and there are whitepapers
[Wednesday 15 September 2010] [05:33:11] <pieterh> but there is nothing lightweight about 0MQ’s internals, really
[Wednesday 15 September 2010] [05:34:06] <mrm2m> ok
[Wednesday 15 September 2010] [05:39:21] <Bruc> hey wat sup all
[Wednesday 15 September 2010] [05:39:51] Part Bruc has left this channel.
[Wednesday 15 September 2010] [05:54:50] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [06:00:48] <kleppari> omarkj: hah, hi!
[Wednesday 15 September 2010] [06:01:15] <omarkj> kleppari: Didn’t notice you here until now, are you guys using 0mq?
[Wednesday 15 September 2010] [06:01:37] <kleppari> I’m experimenting a bit
[Wednesday 15 September 2010] [06:02:23] <kleppari> I just love the idea of message passing for concurrency
[Wednesday 15 September 2010] [06:03:08] <kleppari> well, outside of erlang at least
[Wednesday 15 September 2010] [06:05:14] <kleppari> how about yourself?
[Wednesday 15 September 2010] [06:07:23] <omarkj> Working with it.
[Wednesday 15 September 2010] [06:07:32] <omarkj> Let’s go pm. :)
[Wednesday 15 September 2010] [06:35:00] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [06:54:46] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [07:21:07] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [08:05:56] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [08:29:01] Quit jsimmons has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [08:49:08] <keffo> sustrik, I think I might have figured out the concurrency issue
[Wednesday 15 September 2010] [08:49:38] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [08:49:44] <sustrik> cremes: here i am
[Wednesday 15 September 2010] [08:50:00] <sustrik> keffo: so what’s the problem?
[Wednesday 15 September 2010] [08:51:59] <keffo> I was rewriting (using poll rather than sequential) and to clean up logging etc, and I just noticed one place where I send a null message before sending the rest, except “the rest” already included the incomin route-null, so I suspect it was being sent twie
[Wednesday 15 September 2010] [08:51:59] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [08:52:00] <keffo> twice
[Wednesday 15 September 2010] [08:52:40] <keffo> nullmore, nullmore, uuidmore, data, etc..
[Wednesday 15 September 2010] [08:52:52] <sustrik> ok, i see
[Wednesday 15 September 2010] [08:53:05] <sustrik> so no problem in 0mq to worry about
[Wednesday 15 September 2010] [08:53:10] <keffo> Not at all
[Wednesday 15 September 2010] [08:53:15] <sustrik> :)
[Wednesday 15 September 2010] [08:53:22] <keffo> I never expected that either :)
[Wednesday 15 September 2010] [08:53:48] <sustrik> you are an optimist then :)
[Wednesday 15 September 2010] [08:54:23] <keffo> It’s been very stable and well behaved so far, apart from this issue which is my own fault :)
[Wednesday 15 September 2010] [08:55:46] <keffo> I did 100k pi calcs yesterday, with 10 worker procs on different machines.. the loadbalancer had ~500kb/s passing through it, and in total more than half a gig of logs were generated.. That’s when I decided the logging was way wonky, didnt really tell much
[Wednesday 15 September 2010] [09:01:49] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 15 September 2010] [09:25:04] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [09:25:41] <cremes> sustrik: i’m available for the next hour or so if you want to chat
[Wednesday 15 September 2010] [09:32:23] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [09:45:14] <sustrik> cremes: hi
[Wednesday 15 September 2010] [09:46:04] <sustrik> give me some context first: why have you moved from ruby-ffi to rbzmq?
[Wednesday 15 September 2010] [09:46:49] <cremes> there are multiple ruby runtimes; not all of them support the C extension api that rbzmq uses
[Wednesday 15 September 2010] [09:47:00] <cremes> ffi allows me to support all of the runtimes
[Wednesday 15 September 2010] [09:47:38] <cremes> oh, and i haven’t moved from one to the other; i would just like rbzmq to have the same api as ffi-rzmq
[Wednesday 15 September 2010] [09:48:13] <sustrik> ok, i see
[Wednesday 15 September 2010] [09:48:27] <sustrik> it looks like brian buchanan is not reponding, right?
[Wednesday 15 September 2010] [09:48:39] <cremes> right
[Wednesday 15 September 2010] [09:48:48] <sustrik> ok, i have admin access to the project
[Wednesday 15 September 2010] [09:49:02] <sustrik> i’ll add you there as a developer
[Wednesday 15 September 2010] [09:49:10] <mato> sustrik: cremes: a question; i was wondering myself why there were two ruby bindings
[Wednesday 15 September 2010] [09:49:16] <mato> is there any point in keeping both around?
[Wednesday 15 September 2010] [09:49:20] <sustrik> hi
[Wednesday 15 September 2010] [09:49:37] <mato> if FFI is the way to go then why keep rbzmq at all?
[Wednesday 15 September 2010] [09:50:38] <cremes> um... i don’t know; it’s probably best to get feedback from folks using rbzmq and ask them why they prefer it
[Wednesday 15 September 2010] [09:50:53] <cremes> i’m biased towards the ffi one, obviously ;)
[Wednesday 15 September 2010] [09:51:01] <mato> that might be a good idea; having two bindings is IMO confusing
[Wednesday 15 September 2010] [09:51:03] <sustrik> does it work with any runtime?
[Wednesday 15 September 2010] [09:51:14] <cremes> that’s why i was hoping brian buchanan would speak up
[Wednesday 15 September 2010] [09:51:43] <mato> cremes: is there a mailing list for this? I’ve not seen any discussion on zeromq-dev...
[Wednesday 15 September 2010] [09:52:18] <mato> maybe just ping there, outline the situation (two bindings, ffi works on all runtimes, etc.) and ask the community
[Wednesday 15 September 2010] [09:52:21] <cremes> this has not been brought up on the zerome-dev list
[Wednesday 15 September 2010] [09:52:24] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Wednesday 15 September 2010] [09:52:39] <cremes> i’ll do that
[Wednesday 15 September 2010] [09:53:01] <sustrik> can you explain how does the runtime interoperability work?
[Wednesday 15 September 2010] [09:53:24] <sustrik> so far i had an impression that ffi doesn’t work with all the ruby runtimes
[Wednesday 15 September 2010] [09:53:43] <cremes> sustrik: as of a few days ago, ffi works with all of the runtimes
[Wednesday 15 September 2010] [09:53:48] <sustrik> aha
[Wednesday 15 September 2010] [09:53:50] <cremes> i worked out the last details with the runtime guys
[Wednesday 15 September 2010] [09:53:57] <sustrik> so it’s new
[Wednesday 15 September 2010] [09:54:09] <cremes> the issue with C extensions is that they oftentimes access lots of internal memory structures
[Wednesday 15 September 2010] [09:54:09] <sustrik> that may actually mean that rbzmq is not needed any more
[Wednesday 15 September 2010] [09:54:57] <mato> cremes: what’s the status of FFI support for Ruby MRI (i.e. that which most people are using right now)?
[Wednesday 15 September 2010] [09:54:59] <cremes> ffi allows the runtime authors to hide their implementation details and provide a more solid mechanism for accessing C libraries
[Wednesday 15 September 2010] [09:55:06] <sustrik> i see
[Wednesday 15 September 2010] [09:55:08] <cremes> mato: it’s quite good
[Wednesday 15 September 2010] [09:55:26] <mato> cremes: right, but does “quite good” mean “production quality”?
[Wednesday 15 September 2010] [09:55:57] <mato> i guess what i’m asking is if rbzmq goes away then what will that mean for the current mainstream of Ruby MRI users
[Wednesday 15 September 2010] [09:56:11] <mato> will they all just happily start using the FFI binding, or not?
[Wednesday 15 September 2010] [09:56:12] <cremes> mato: perhaps; i can only speak for how it works with 0mq; obviously we still have hangs until 2.1.x comes out with EINTR support for blocking calls
[Wednesday 15 September 2010] [09:56:39] <mato> sure, but the EINTR hangs are orthogonal to FFI/non-FFI
[Wednesday 15 September 2010] [09:56:41] <sustrik> well, whatever happens the rbzmq project is not going to be killed
[Wednesday 15 September 2010] [09:56:44] <sustrik> so no worry
[Wednesday 15 September 2010] [09:56:46] <cremes> mato: i’ll ask for feedback on the ML
[Wednesday 15 September 2010] [09:57:04] <sustrik> the problem is that it doesn’t have a permanent maintainer
[Wednesday 15 September 2010] [09:57:18] <cremes> mato: sure, it’s orthogonal, but it’s hard to say if there are other bugs when you get a hang
[Wednesday 15 September 2010] [09:57:25] <cremes> is it due to the blocking behavior or something else?
[Wednesday 15 September 2010] [09:57:31] <mato> well then put a note up about that (lack of maintainer) on the rbzmq project page
[Wednesday 15 September 2010] [09:57:39] <cremes> i don’t have the time or energy to run stuff under gdb all of the time to figure that out :)
[Wednesday 15 September 2010] [09:58:07] <cremes> let’s take this to the ML and see what users have to say
[Wednesday 15 September 2010] [09:58:10] <mato> yup
[Wednesday 15 September 2010] [09:58:12] <sustrik> ok
[Wednesday 15 September 2010] [09:58:18] <cremes> btw, from a search on github most projects are using the ffi-based binding
[Wednesday 15 September 2010] [09:58:36] <cremes> but i certainly don’t want to lose or alienate the rbzmq users
[Wednesday 15 September 2010] [09:59:01] <cremes> which takes us full circle; i’d like each one to present the same api to the user (based off of the C and C++ binding apis)
[Wednesday 15 September 2010] [09:59:11] <mato> definitely
[Wednesday 15 September 2010] [09:59:22] <mato> otherwise it just all gets way too confusing
[Wednesday 15 September 2010] [09:59:26] <cremes> if people can switch back and forth with no code changes, that’s a great situation for the community
[Wednesday 15 September 2010] [09:59:33] <mato> is it much work to synchronize the APIs?
[Wednesday 15 September 2010] [10:00:02] <cremes> mato: not a lot but unfortunately i don’t have the C chops to do it myself
[Wednesday 15 September 2010] [10:00:12] <cremes> so i need someone else to agree to it
[Wednesday 15 September 2010] [10:00:17] <cremes> to do the work
[Wednesday 15 September 2010] [10:00:42] <mato> cremes: well, ask for help on the list and we’ll see what happens
[Wednesday 15 September 2010] [10:00:53] <cremes> will do
[Wednesday 15 September 2010] [10:01:08] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [10:05:30] <sustrik> mato: man bug!
[Wednesday 15 September 2010] [10:05:44] <sustrik> zmq_socket(3):
[Wednesday 15 September 2010] [10:05:47] <sustrik> “When a ZMQ_XREQ socket is connected to a ZMQ_REP socket each message sent must consist of an empty message part, the delimiter, followed by one or more body parts.”
[Wednesday 15 September 2010] [10:05:48] <mato> sustrik: !!?!? :-)
[Wednesday 15 September 2010] [10:06:22] <sustrik> the identities should be mentioned
[Wednesday 15 September 2010] [10:06:48] <sustrik> they are definitely mentioned for ZMQ_XREP
[Wednesday 15 September 2010] [10:06:52] <ptrb> yeah, it’s not really clear to me what work i have to do at the app level to do XREQ/REP (or REQ/XREP)
[Wednesday 15 September 2010] [10:07:09] <mato> uh, yeah, i kind of punted on explaining the identity stack on the XREQ side at the time
[Wednesday 15 September 2010] [10:07:20] <mato> sustrik: that stuff was written in a hurry, you remember :)
[Wednesday 15 September 2010] [10:07:23] <sustrik> ptrb: there was a nice diagram somewhere...
[Wednesday 15 September 2010] [10:07:39] <mato> sustrik: anyway, it’s not a bug, it’s just a simplification :)
[Wednesday 15 September 2010] [10:07:43] <mato> sustrik: i’ll look into it.
[Wednesday 15 September 2010] [10:08:39] <cremes> if not, tell me what is unclear and i’ll fix it (or edit the page yourself)
[Wednesday 15 September 2010] [10:08:40] <ptrb> clicks
[Wednesday 15 September 2010] [10:08:52] <sustrik> ah – that’s the one i meant
[Wednesday 15 September 2010] [10:08:55] <cremes> i was certainly confused by that stuff which is why i wrote it down
[Wednesday 15 September 2010] [10:09:14] <ptrb> yeah, it definitely helps
[Wednesday 15 September 2010] [10:09:23] <ptrb> but ruby is definitely not the right language to use in examples like this
[Wednesday 15 September 2010] [10:09:47] <ptrb> it’s as opaque as perl without the benefit of perl’s saturation
[Wednesday 15 September 2010] [10:09:53] <cremes> ha
[Wednesday 15 September 2010] [10:10:03] <mato> :-)
[Wednesday 15 September 2010] [10:10:37] * mato will think of some suitably terse way of describing the identity stack in zmq_socket(3)
[Wednesday 15 September 2010] [10:36:52] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 15 September 2010] [10:45:23] <CIA-20> zeromq2: 03Martin Sustrik 07maint * re2802d9 10/ src/options.cpp : values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9lAWvv[Wednesday 15 September 2010] [10:46:54] <pieterh> sustrik: nice to see the validation of option values
[Wednesday 15 September 2010] [10:47:45] <sustrik> there are 3 bugs filled by cremes :)
[Wednesday 15 September 2010] [10:47:53] <mato> sustrik: incidentally, there’s one annoying thing regarding the option values
[Wednesday 15 September 2010] [10:48:00] <sustrik> yes?
[Wednesday 15 September 2010] [10:48:14] <mato> well, it’s the whole “option types” issue
[Wednesday 15 September 2010] [10:48:24] <sustrik> ?
[Wednesday 15 September 2010] [10:48:26] <mato> one thing that i realised the other day when i was writing some simple test cases
[Wednesday 15 September 2010] [10:48:47] <mato> is that in order to use options sensibly from C, you have to have uint64_t / int64_t defined
[Wednesday 15 September 2010] [10:49:03] <mato> now, obviously those are not defined by zmq.h
[Wednesday 15 September 2010] [10:49:25] <mato> of course if you want to use the API you have to do “platform specific stuff” to get those types
[Wednesday 15 September 2010] [10:49:31] <mato> not a good situation :-(
[Wednesday 15 September 2010] [10:50:27] <mato> especially on windows where M$ does not ship stdint.h
[Wednesday 15 September 2010] [10:50:36] <sustrik> i know
[Wednesday 15 September 2010] [10:50:54] <sustrik> what can i do without breaking the backward compatibility?
[Wednesday 15 September 2010] [10:51:15] <mato> probably nothing
[Wednesday 15 September 2010] [10:51:20] <sustrik> :|
[Wednesday 15 September 2010] [10:51:54] <mato> hmm, maybe...
[Wednesday 15 September 2010] [10:52:37] <mato> well, in theory you could define
zmq_ namespaced equivalents of those types including the platform magic in zmq.h, but that’s kind of ugly
[Wednesday 15 September 2010] [10:52:52] <mato> and it’d probably still break C++ due to it’s strict notions of types
[Wednesday 15 September 2010] [10:52:56] <sustrik> it has to be solved properly
[Wednesday 15 September 2010] [10:53:04] <sustrik> BSD-style
[Wednesday 15 September 2010] [10:53:13] <mato> hmm
[Wednesday 15 September 2010] [10:53:17] <sustrik> 4-byte unsinged integer in network byte order
[Wednesday 15 September 2010] [10:53:20] <mato> hang on
[Wednesday 15 September 2010] [10:53:21] <sustrik> and such
[Wednesday 15 September 2010] [10:53:22] <mato> what?
[Wednesday 15 September 2010] [10:53:30] <mato> what does network byte order have to do with it? :)
[Wednesday 15 September 2010] [10:53:45] <sustrik> that’s what i do when usign BSD sockets
[Wednesday 15 September 2010] [10:53:52] <mato> ?
[Wednesday 15 September 2010] [10:54:10] <sustrik> addr.port = htons (port);
[Wednesday 15 September 2010] [10:55:56] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [10:58:19] <mato> sustrik: ah, you’re right, i didn’t realise e.g. sockaddr_in.sin_port was in network byte order
[Wednesday 15 September 2010] [10:58:32] <mato> sustrik: the thing is, the setsockopt stuff is somewhat ad-hoc
[Wednesday 15 September 2010] [10:58:43] <CIA-20> zeromq2: 03Martin Sustrik 07master * re2802d9 10/ src/options.cpp : values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9lAWvv[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: 03Martin Sustrik 07master * rff10807 10/ src/options.cpp :
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: * maint:
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9P99Fu[Wednesday 15 September 2010] [10:58:48] <mato> it does not use network byte order AFAIK
[Wednesday 15 September 2010] [10:59:18] <mato> sustrik: anyway thinking about it, i guess zmq.h will have to define its own types
[Wednesday 15 September 2010] [11:00:14] <mato> the main problem with that is doing it in a portable fashion :-(
[Wednesday 15 September 2010] [11:07:22] <ptrb> #defines, #defines for everybody!!
[Wednesday 15 September 2010] [11:07:58] <mato> sure, feel free to suggest how to portably define a 64-bit unsigned integer type :-)
[Wednesday 15 September 2010] [11:08:20] <ptrb> you just take two ints and mash ‘em together, obviously
[Wednesday 15 September 2010] [11:08:27] <mato> :-)
[Wednesday 15 September 2010] [11:29:09] <sustrik> mato: btw, there’s some packaging PATCH on the mailing list
[Wednesday 15 September 2010] [11:29:15] <sustrik> will you reply to that?
[Wednesday 15 September 2010] [11:30:30] <mato> sustrik: yes, later, i know it’s there
[Wednesday 15 September 2010] [11:30:32] <mato> have to go now
[Wednesday 15 September 2010] [11:30:45] <sustrik> ok, there’s on in the bug tracker as well
[Wednesday 15 September 2010] [11:30:51] <mato> sustrik: i’d have just applied it but the license stuff needs to be sorted out
[Wednesday 15 September 2010] [11:31:05] <mato> else all i can reply is “please state your patch is licensed under...”
[Wednesday 15 September 2010] [11:31:14] <mato> have discussed with pieter will update you this evening
[Wednesday 15 September 2010] [11:31:18] <mato> or pieter will
[Wednesday 15 September 2010] [11:31:19] <mato> bbl
[Wednesday 15 September 2010] [11:31:32] <sustrik> should i ask for the license?
[Wednesday 15 September 2010] [11:31:46] <mato> don’t bother lets just fix it
[Wednesday 15 September 2010] [11:31:51] <mato> talk to you in the evening
[Wednesday 15 September 2010] [11:31:54] <sustrik> ok
[Wednesday 15 September 2010] [11:40:49] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [11:41:58] <Samy> sustrik, just had some questions regarding zeromq’s lock-less data structures.
[Wednesday 15 September 2010] [11:42:27] <sustrik> yes?
[Wednesday 15 September 2010] [11:42:51] <Samy> sustrik, I’m working on a library to help ease concurrent programming, it includes a plethora of concurrent data structures, synchronization methods and mechanisms for SMR (currently using hazard pointers, considering implementing RCU).
[Wednesday 15 September 2010] [11:43:37] <Samy> sustrik, I was curious if there were some features you would really like to see, and some specific constrainted data structures (M:N consumer/producers) for zeromq.
[Wednesday 15 September 2010] [11:43:52] <Samy> sustrik, I was also curious if SMR was important for ZeroMQ at all.
[Wednesday 15 September 2010] [11:44:38] <sustrik> sorry, what’s SMR?
[Wednesday 15 September 2010] [11:44:48] <Samy> sustrik, safe memory reclamation.
[Wednesday 15 September 2010] [11:45:26] <Samy> sustrik, usually it’s important for unbunded lock-less data structures (simple stack being a great example).
[Wednesday 15 September 2010] [11:46:48] <sustrik> does it translate to “make the current state of the memory visible to the other CPU codes”?
[Wednesday 15 September 2010] [11:47:23] <sustrik> sorry, i am not an expert on lock-free algos
[Wednesday 15 September 2010] [11:48:03] <Samy> sustrik, well, I guess fundamentally, it’s more like “let other CPUs know I am not using this memory”.
[Wednesday 15 September 2010] [11:48:27] <sustrik> ah, ok, let me explain what 0mq is doing
[Wednesday 15 September 2010] [11:48:28] <Samy> sustrik, some techniques will require a full barrier (RCU) while others do not (hazard pointers).
[Wednesday 15 September 2010] [11:48:41] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [11:48:57] <sustrik> the communication always happens between exactly two endpoins
[Wednesday 15 September 2010] [11:49:03] <sustrik> ie. at most 2 cpu cores
[Wednesday 15 September 2010] [11:49:25] <sustrik> so each lock-free queue has exactly one writer and exactly one reader
[Wednesday 15 September 2010] [11:49:30] <Samy> Ah, makes it much easier. :-)
[Wednesday 15 September 2010] [11:49:36] <Samy> Sorry, let me pop open the source-code.
[Wednesday 15 September 2010] [11:49:48] <sustrik> src/ypipe.hpp
[Wednesday 15 September 2010] [11:49:52] <Samy> Thanks, loading.
[Wednesday 15 September 2010] [11:50:26] <sustrik> what it does basically is that writer is appending new items to the linked list
[Wednesday 15 September 2010] [11:50:39] <sustrik> the reader is reading items from the linked list
[Wednesday 15 September 2010] [11:50:54] <Samy> I don’t see any padding, sustrik.
[Wednesday 15 September 2010] [11:51:08] <sustrik> what padding?
[Wednesday 15 September 2010] [11:51:09] <Samy> Between w and r.
[Wednesday 15 September 2010] [11:51:25] <Samy> sustrik, to prevent false cache line sharing, this improves concurrency.
[Wednesday 15 September 2010] [11:51:36] <sustrik> that would be nice
[Wednesday 15 September 2010] [11:51:49] <Samy> sustrik, it can be drastic. :-)
[Wednesday 15 September 2010] [11:51:53] <Samy> Let me show you a simple example on this machine.
[Wednesday 15 September 2010] [11:52:57] <sustrik> ok, so what has to be done is separate the variables that belong to the reader and those that belong to the writer
[Wednesday 15 September 2010] [11:53:04] <sustrik> and keep them in different cachelines
[Wednesday 15 September 2010] [11:53:06] <sustrik> right?
[Wednesday 15 September 2010] [11:53:27] <keffo> the data too
[Wednesday 15 September 2010] [11:53:48] <Samy> [
sbahra@sbahra validate]$ time ./ck_fifo_spsc 8 1 100000
[Wednesday 15 September 2010] [11:53:49] <Samy> real 0m5.294s
[Wednesday 15 September 2010] [11:53:59] <Samy> [
sbahra@sbahra validate]$ time ./ck_fifo_spsc 8 1 100000
[Wednesday 15 September 2010] [11:53:59] <Samy> real 0m7.718s
[Wednesday 15 September 2010] [11:54:12] <Samy> sustrik, the latter is without appropriate padding.
[Wednesday 15 September 2010] [11:54:16] <sustrik> nice
[Wednesday 15 September 2010] [11:54:26] <sustrik> the data are allocated in a contiguous block
[Wednesday 15 September 2010] [11:54:37] <keffo> that was pretty substantial, what arch is that?
[Wednesday 15 September 2010] [11:54:54] <sustrik> to avoid excessive memory allocation
[Wednesday 15 September 2010] [11:54:59] <Samy> sustrik, that is a simple benchmark that creates a token ring using a single-producer/single-consumer lock-less queue (passes around 100000 across 8 threads some number of iterations)
[Wednesday 15 September 2010] [11:55:05] <Samy> keffo, that is on a Nehalem box.
[Wednesday 15 September 2010] [11:55:09] <keffo> aho
[Wednesday 15 September 2010] [11:55:25] <keffo> sustrik, You should look into pooling, not only for that
[Wednesday 15 September 2010] [11:55:40] <Samy> sustrik, you want the writer and reader variables to be on a seperate cache line.
[Wednesday 15 September 2010] [11:55:56] <sustrik> right, i understand that
[Wednesday 15 September 2010] [11:55:56] <Samy> sustrik, so for example, void
*reader; char pad[56]; void
*writer; ...
[Wednesday 15 September 2010] [11:56:00] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [11:56:12] <sustrik> not sure how to do that in portable fashion though
[Wednesday 15 September 2010] [11:56:22] <keffo> compile time :)
[Wednesday 15 September 2010] [11:56:28] <keffo> macro hell
[Wednesday 15 September 2010] [11:56:29] <ptrb> did someone say #define??
[Wednesday 15 September 2010] [11:56:32] <ptrb> :D
[Wednesday 15 September 2010] [11:56:34] <Samy> sustrik, not very portable (my library tries to make that portable by doing the hard work of generating those constants at compile-time).
[Wednesday 15 September 2010] [11:56:34] <sustrik> :)
[Wednesday 15 September 2010] [11:56:49] <Samy> sustrik, but modern IA32 and SPARCv9 boxes I know of all have 64 byte cache lines.
[Wednesday 15 September 2010] [11:56:57] <sustrik> well, if we assumed the cache line size is 64 bytes
[Wednesday 15 September 2010] [11:56:59] <keffo> cpuid fiddling works I guess :)
[Wednesday 15 September 2010] [11:57:03] <sustrik> we would be right in most cases
[Wednesday 15 September 2010] [11:57:06] <sustrik> no?
[Wednesday 15 September 2010] [11:57:14] <keffo> all x86_64 is >64 no?
[Wednesday 15 September 2010] [11:57:21] <sustrik> 128?
[Wednesday 15 September 2010] [11:57:22] <Samy> keffo, no.
[Wednesday 15 September 2010] [11:57:25] <Samy> keffo, 64.
[Wednesday 15 September 2010] [11:57:38] <Samy> keffo, when I mean IA32, I mean x86/x86_64.
[Wednesday 15 September 2010] [11:57:41] <keffo> >=64 then :)
[Wednesday 15 September 2010] [11:57:47] <Samy> I don’t know of any > 64.
[Wednesday 15 September 2010] [11:58:18] <sustrik> padding to 64 bytes seems reasonable imo
[Wednesday 15 September 2010] [11:58:34] <keffo> I meant, x86_64 is presumably >=64, while x86 is not
[Wednesday 15 September 2010] [11:58:50] <sustrik> that would be 64 bytes per reader and 64 bytes per writer
[Wednesday 15 September 2010] [11:59:06] <sustrik> the shared varaible should probably be on a separate cahceline
[Wednesday 15 September 2010] [11:59:08] <keffo> Samy, Have you tried measuring on itanium? :)
[Wednesday 15 September 2010] [11:59:16] <sustrik> so it’s 192 bytes per queue
[Wednesday 15 September 2010] [11:59:40] <Samy> keffo, Itanium is very interesting, but barely used, so no.
[Wednesday 15 September 2010] [12:00:06] <Samy> keffo, I would like to work on a port of my library there but I have had design issues, just recently did a major design change.
[Wednesday 15 September 2010] [12:00:07] <keffo> Yeah, void of any cachemisses it should be
[Wednesday 15 September 2010] [12:00:20] <Samy> sustrik, sorry, can you continue with your explanation of ypipe?
[Wednesday 15 September 2010] [12:00:25] <sustrik> ah
[Wednesday 15 September 2010] [12:00:42] * keffo butts out :)
[Wednesday 15 September 2010] [12:00:42] <sustrik> ok, so part of the list belongs to the writer thread
[Wednesday 15 September 2010] [12:00:50] <sustrik> part of it to the reader thread
[Wednesday 15 September 2010] [12:01:13] <sustrik> the only point where the two interact is when reader has no more data to read
[Wednesday 15 September 2010] [12:01:21] <sustrik> then it does CAS
[Wednesday 15 September 2010] [12:01:33] <sustrik> to get the writer’s portion of the list
[Wednesday 15 September 2010] [12:02:01] <Samy> So, basically, you batch the operations?
[Wednesday 15 September 2010] [12:02:06] <sustrik> exactly
[Wednesday 15 September 2010] [12:02:09] <sustrik> that’s the key
[Wednesday 15 September 2010] [12:02:14] <Samy> So, writer does enqueue operations to the queue.
[Wednesday 15 September 2010] [12:02:20] <sustrik> ack
[Wednesday 15 September 2010] [12:02:21] <Samy> The reader occassionally does a batch dequeue.
[Wednesday 15 September 2010] [12:02:24] <sustrik> ack
[Wednesday 15 September 2010] [12:02:27] <Samy> Ok.
[Wednesday 15 September 2010] [12:02:48] <Samy> sustrik, why do you use CAS for the batch dequeue? That can be implemented using an xchg, making the batch dequeue a wait-free operation.
[Wednesday 15 September 2010] [12:02:48] <sustrik> it’s pretty effective just because of the batching
[Wednesday 15 September 2010] [12:03:24] <sustrik> hm, i haven’t seen the code for a long time
[Wednesday 15 September 2010] [12:03:27] <sustrik> let me have a look
[Wednesday 15 September 2010] [12:04:02] <Samy> It looks like you do a single CAS.
[Wednesday 15 September 2010] [12:04:12] <Samy> Line 137, check_read (I assume?)
[Wednesday 15 September 2010] [12:04:56] * Samy looks at yqueue
[Wednesday 15 September 2010] [12:05:01] <sustrik> “If there are no
[Wednesday 15 September 2010] [12:05:01] <sustrik> // items to prefetch, set c to NULL”
[Wednesday 15 September 2010] [12:05:09] <sustrik> Samy: yes
[Wednesday 15 September 2010] [12:05:27] <sustrik> the cas is used to communicate back to the writer
[Wednesday 15 September 2010] [12:05:28] <Samy> You can implement this without any lock-free operations.
[Wednesday 15 September 2010] [12:05:43] * sustrik is listening
[Wednesday 15 September 2010] [12:06:32] <sustrik> you mean without atomic ops?
[Wednesday 15 September 2010] [12:06:41] <sustrik> or bus locking?
[Wednesday 15 September 2010] [12:07:14] <Samy> Without explicit atomic operations.
[Wednesday 15 September 2010] [12:07:40] <Samy> sustrik, the idea is to share a stub node for both tail and head of the queue.
[Wednesday 15 September 2010] [12:08:16] <sustrik> can you explain in more detail?
[Wednesday 15 September 2010] [12:08:17] <Samy> sustrik, writer updates tail, always updating the next pointer, and reader always assume first node is stub entry.
[Wednesday 15 September 2010] [12:08:31] <Samy> sustrik, I can show you source-code.
[Wednesday 15 September 2010] [12:08:39] <Samy> Let me see if I find it, hold on.
[Wednesday 15 September 2010] [12:09:02] <sustrik> ok
[Wednesday 15 September 2010] [12:09:35] <sustrik> how does the communication between two cpu cores happens then?
[Wednesday 15 September 2010] [12:09:41] <sustrik> cache coherency algos?
[Wednesday 15 September 2010] [12:11:21] <Samy> sustrik, yes.
[Wednesday 15 September 2010] [12:11:35] <Samy> sustrik, the point is, the only time we need to really share state is if the queue is empty.
[Wednesday 15 September 2010] [12:11:43] <sustrik> ack
[Wednesday 15 September 2010] [12:11:50] <Samy> sustrik, we need a way to detect if the queue is empty atomically and update both head and tail atomically.
[Wednesday 15 September 2010] [12:11:54] <Samy> sustrik, by sharing a stub node, this is possible.
[Wednesday 15 September 2010] [12:12:23] <sustrik> i think something like that is done in ypipe
[Wednesday 15 September 2010] [12:12:33] <sustrik> there’s an emty item in the list
[Wednesday 15 September 2010] [12:12:40] <sustrik> that servers as a placeholder
[Wednesday 15 September 2010] [12:12:50] <sustrik> that’s what you mean by stub, right?
[Wednesday 15 September 2010] [12:12:52] <Samy> Yes.
[Wednesday 15 September 2010] [12:13:20] <Samy> This isn’t fenced correctly, but that should be fine on IA32.
[Wednesday 15 September 2010] [12:14:08] <sustrik> i thought that IA requires you to fence explicitely
[Wednesday 15 September 2010] [12:14:23] <sustrik> ah, you mean x85
[Wednesday 15 September 2010] [12:14:25] <sustrik> 86
[Wednesday 15 September 2010] [12:14:57] <Samy> Yes.
[Wednesday 15 September 2010] [12:15:25] <Samy> The fences may be sufficient, I just haven’t verified it on non-IA32 yet. :-)
[Wednesday 15 September 2010] [12:15:25] <sustrik> ok, there’s one more issue there
[Wednesday 15 September 2010] [12:15:48] <sustrik> when reader finds out that there are no more items to read
[Wednesday 15 September 2010] [12:15:53] <sustrik> it goes asleep
[Wednesday 15 September 2010] [12:16:07] <sustrik> it’s writers responsibity to wake it up
[Wednesday 15 September 2010] [12:16:29] <sustrik> so the writer has to be informed about the fact that reader tried to get more items and failed
[Wednesday 15 September 2010] [12:16:41] <Samy> sustrik, futex(2) works well.
[Wednesday 15 September 2010] [12:16:49] <sustrik> yes, it’s similar
[Wednesday 15 September 2010] [12:17:10] <sustrik> but keep in mind that this is a multi-platform app
[Wednesday 15 September 2010] [12:17:47] <Samy> Things sort of suck without futex. :-(
[Wednesday 15 September 2010] [12:18:01] <Samy> (or a similar mechanism, at least in ring-3)
[Wednesday 15 September 2010] [12:18:08] <sustrik> that’s how it is :|
[Wednesday 15 September 2010] [12:18:28] <Samy> But you could abstract a generic CV layer that uses futex directly if available.
[Wednesday 15 September 2010] [12:18:40] <Samy> sustrik, where is this wake-up mechanism implemented?
[Wednesday 15 September 2010] [12:18:59] <sustrik> out of the class, it’s a dumb socket pair
[Wednesday 15 September 2010] [12:19:09] <sustrik> i had a futex implementation once
[Wednesday 15 September 2010] [12:19:15] <Samy> What happened?
[Wednesday 15 September 2010] [12:19:22] <sustrik> but it was pain in the ass to make it work everywhere
[Wednesday 15 September 2010] [12:19:28] <Samy> I see.
[Wednesday 15 September 2010] [12:19:32] <sustrik> some linux kernels pretend to have futexes
[Wednesday 15 September 2010] [12:19:46] <sustrik> but astucally return ENOTSUP when you try to use them
[Wednesday 15 September 2010] [12:19:47] <sustrik> and alike
[Wednesday 15 September 2010] [12:19:57] <Samy> I see.
[Wednesday 15 September 2010] [12:20:14] <sustrik> anyway, the wake up mechanism is irrelevant for this discussion
[Wednesday 15 September 2010] [12:20:31] <Samy> Mostly, yes.
[Wednesday 15 September 2010] [12:20:36] <sustrik> what’s relevant is the writer has to be notified that reader is sleeping
[Wednesday 15 September 2010] [12:20:46] <sustrik> that’s what the second cas is for
[Wednesday 15 September 2010] [12:20:55] <sustrik> in flush function
[Wednesday 15 September 2010] [12:21:17] <Samy> I don’t understand.
[Wednesday 15 September 2010] [12:21:30] <Samy> Why not simply use plain loads and stores if this is single reader and consumer?
[Wednesday 15 September 2010] [12:22:06] <sustrik> because one bit of information is passed the other way round
[Wednesday 15 September 2010] [12:22:10] <sustrik> from reader to writer
[Wednesday 15 September 2010] [12:22:19] <sustrik> “i am sleeping”
[Wednesday 15 September 2010] [12:22:28] <sustrik> that’s when c is set to NULL
[Wednesday 15 September 2010] [12:22:50] <sustrik> maybe it can be done without atomic ops, i am not an expert
[Wednesday 15 September 2010] [12:23:20] <sustrik> however, if it was a normal locking code, this would be a place where races can occur
[Wednesday 15 September 2010] [12:24:12] <Samy> Ok.
[Wednesday 15 September 2010] [12:24:40] <sustrik> so, reader tries to get the latest batch of the items
[Wednesday 15 September 2010] [12:24:51] <sustrik> and if there are none, it sets c to NULL
[Wednesday 15 September 2010] [12:25:29] <sustrik> in flush function, writer adds new batch of items and if it finds out that c was NULL previously
[Wednesday 15 September 2010] [12:25:35] <sustrik> it knows the reader is sleeping
[Wednesday 15 September 2010] [12:25:42] <sustrik> and that it should wake it up
[Wednesday 15 September 2010] [12:26:11] <sustrik> my feeling is that the operations on c have to be atomic
[Wednesday 15 September 2010] [12:26:18] <sustrik> not 100% sure though
[Wednesday 15 September 2010] [12:30:42] <Samy> read_asleep = false; loop { old = xchg(top, NULL); if old == null then reader_asleep = true; sleep(); }
[Wednesday 15 September 2010] [12:31:25] <Samy> Sorry, read_asleep is in loop.
[Wednesday 15 September 2010] [12:32:40] <sustrik> that’s the writer side?
[Wednesday 15 September 2010] [12:33:50] <Samy> Reader.
[Wednesday 15 September 2010] [12:34:09] <Samy> There is still a risk of spurious wake-up, but that isn’t a big deal.
[Wednesday 15 September 2010] [12:34:17] <Samy> If it is, then you basically implement a barrier.
[Wednesday 15 September 2010] [12:34:55] <Samy> You have a reader and a writer shared variable. Writer signals reader once, only if reader has signaled writer since last wake-up.
[Wednesday 15 September 2010] [12:35:55] <sustrik> yes, that’s the goal
[Wednesday 15 September 2010] [12:36:03] <Samy> And the algorithm.
[Wednesday 15 September 2010] [12:36:15] * sustrik is a dummy
[Wednesday 15 September 2010] [12:36:21] <Samy> Me too.
[Wednesday 15 September 2010] [12:36:28] <sustrik> can you give a pseudo-code for writer side as well?
[Wednesday 15 September 2010] [12:38:19] <sustrik> in the code above reader_asleep is another shared variable, right?
[Wednesday 15 September 2010] [12:39:32] <pieterh> sustrik: I was trying to say something about options but got interrupted by stuff...
[Wednesday 15 September 2010] [12:40:02] <pieterh> API should IMO reject an identity starting with zero byte
[Wednesday 15 September 2010] [12:40:19] <sustrik> now you are likely to get interrupted by brain-damaging lock-free algo discussion :)
[Wednesday 15 September 2010] [12:40:33] <pieterh> :-/
[Wednesday 15 September 2010] [12:40:34] <sustrik> yes, that would be nice
[Wednesday 15 September 2010] [12:41:04] <Samy> reader: loop { r = false; store_fence; w = true; o = xchg(top, NULL); if old == null then { r = true; sleep(); } }
[Wednesday 15 September 2010] [12:41:05] <Samy> writer: add(); c_w = w; load_fence; c_r = r; if c_r == true and c_w == true; then { c_w = false; signal_reader; }
[Wednesday 15 September 2010] [12:41:24] <sustrik> pieter: fancy sumbitting a patch?
[Wednesday 15 September 2010] [12:41:49] <pieterh> sustrik: I’m pulling master as we speak, yes I’ll make you a patch
[Wednesday 15 September 2010] [12:42:45] <sustrik> thanks
[Wednesday 15 September 2010] [12:42:56] <sustrik> Samy: sorry, having 3 discussions in parallel
[Wednesday 15 September 2010] [12:43:34] <Samy> Hey, no problem. :-)
[Wednesday 15 September 2010] [12:43:39] <Samy> I’ll be back in some minutes.
[Wednesday 15 September 2010] [12:43:53] <sustrik> sure, checking your code right now
[Wednesday 15 September 2010] [12:46:14] <sustrik> what’s top?
[Wednesday 15 September 2010] [12:46:50] <sustrik> which variable belong to whom?
[Wednesday 15 September 2010] [12:46:57] <sustrik> both r and w seem to be shared
[Wednesday 15 September 2010] [12:48:23] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 15 September 2010] [12:49:34] <pieterh> sustrik: np. options.cpp already checks that... :-)
[Wednesday 15 September 2010] [12:49:43] <sustrik> good
[Wednesday 15 September 2010] [12:51:56] <Samy> sustrik, yes, they are shared.
[Wednesday 15 September 2010] [12:52:46] <Samy> sustrik, that statement is whatever condition you use to check if the queue is empty.
[Wednesday 15 September 2010] [12:53:17] <sustrik> this one: o = xchg(top, NULL); ?
[Wednesday 15 September 2010] [12:53:25] <Samy> That and next.
[Wednesday 15 September 2010] [12:53:34] <Samy> o == null
[Wednesday 15 September 2010] [12:53:35] <Samy> (not old)
[Wednesday 15 September 2010] [12:54:11] <sustrik> do c_r and c_w are local to writer
[Wednesday 15 September 2010] [12:54:17] <Samy> Yes.
[Wednesday 15 September 2010] [12:54:32] * sustrik is trying to make sense of it
[Wednesday 15 September 2010] [12:54:58] <Samy> The reason we have 2 variables is that writer will signal the reader only once.
[Wednesday 15 September 2010] [12:55:40] <Samy> The reader will then wake-up, and if the queue is empty it will indicate that it is asleep and it is fine for the writer to send another signal.
[Wednesday 15 September 2010] [12:56:22] <Samy> sustrik, if you treat it like a stack, it will make more sense.
[Wednesday 15 September 2010] [12:56:50] <sustrik> Samy: stack of what?
[Wednesday 15 September 2010] [12:56:54] <Samy> sustrik, for a FIFO, you can use another technique.
[Wednesday 15 September 2010] [12:57:04] <Samy> sustrik, of objects.
[Wednesday 15 September 2010] [12:57:11] <sustrik> that’s the case in 0MQ
[Wednesday 15 September 2010] [12:57:20] <Samy> sustrik, this is in reference of xchg(top, ...) :)
[Wednesday 15 September 2010] [12:57:32] <Samy> sustrik, what’s of relevance to you is the notion of the r and w variables, that’s all.
[Wednesday 15 September 2010] [12:58:03] <sustrik> r = “i am sleeeping”
[Wednesday 15 September 2010] [12:58:14] <sustrik> what abour w?
[Wednesday 15 September 2010] [12:58:19] <sustrik> about*
[Wednesday 15 September 2010] [12:58:39] <Samy> w = “reader has woken-up since the last signal”
[Wednesday 15 September 2010] [12:59:20] <sustrik> soemging is missing
[Wednesday 15 September 2010] [12:59:25] <sustrik> w is never set to false
[Wednesday 15 September 2010] [12:59:34] <Samy> Oh, sorry.
[Wednesday 15 September 2010] [12:59:44] <Samy> { c_w = false; signal_reader; } should be { w = false; signal_reader; }
[Wednesday 15 September 2010] [13:00:14] <sustrik> lk,now it makes sense
[Wednesday 15 September 2010] [13:00:56] <sustrik> now, what about the fences
[Wednesday 15 September 2010] [13:01:29] <sustrik> does it sync the data in the queue as well?
[Wednesday 15 September 2010] [13:02:03] <Samy> You can figure that out.
[Wednesday 15 September 2010] [13:02:22] <Samy> That is mainly meant for those flags.
[Wednesday 15 September 2010] [13:02:29] Quit AndrewBC has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [13:02:56] <Samy> The idea, reader should always appear to be set to false before writer is set to true (it must indicate it has woken up before it indicates it is fine to send another signal).
[Wednesday 15 September 2010] [13:03:42] <sustrik> got it
[Wednesday 15 September 2010] [13:04:14] <sustrik> that part ensures that everything happens in nice step-lock fashion
[Wednesday 15 September 2010] [13:04:19] <Samy> Right.
[Wednesday 15 September 2010] [13:04:42] <sustrik> now few technical questions...
[Wednesday 15 September 2010] [13:04:49] <Samy> I’m no expert, but ok.
[Wednesday 15 September 2010] [13:05:02] <sustrik> what does lock;xchg() actually means
[Wednesday 15 September 2010] [13:05:11] <sustrik> is there an implied barrier?
[Wednesday 15 September 2010] [13:05:29] <Samy> On IA32, atomic operations have a total order across all processors.
[Wednesday 15 September 2010] [13:05:46] <Samy> There is an implicit pipeline flush.
[Wednesday 15 September 2010] [13:06:11] <Samy> There is an implied barrier.
[Wednesday 15 September 2010] [13:06:24] <Samy> There is also no need for the lock prefix on xchg.
[Wednesday 15 September 2010] [13:06:41] <Samy> xchg is guaranteed to be atomic (which is what also makes it expensive to use, if you don’t need it).
[Wednesday 15 September 2010] [13:06:42] <sustrik> in that case there’s no global ordering, right?
[Wednesday 15 September 2010] [13:06:52] <Samy> Usually, yes.
[Wednesday 15 September 2010] [13:07:10] <Samy> add is not guaranteed to be atomic, lock add is.
[Wednesday 15 September 2010] [13:07:16] <Samy> xchg is always atomic.
[Wednesday 15 September 2010] [13:07:21] <sustrik> oh my
[Wednesday 15 September 2010] [13:07:41] <Samy> It’s expensive compared to simple loads and stores, that’s for sure.
[Wednesday 15 September 2010] [13:07:41] <sustrik> so there’s an implied barrier on xchg, right?
[Wednesday 15 September 2010] [13:07:45] <Samy> Yes.
[Wednesday 15 September 2010] [13:07:58] <sustrik> then, in your code we have 2 barriers
[Wednesday 15 September 2010] [13:08:03] <Samy> For more information, you can see Volume 3 of the Intel Architecture Manuals, Chapter 8.
[Wednesday 15 September 2010] [13:08:37] <sustrik> isn’t it better to use single cas with a barrier then?
[Wednesday 15 September 2010] [13:08:38] <Samy> sustrik, you don’t need any of those barriers for IA32. Stores are always seen in order.
[Wednesday 15 September 2010] [13:09:19] <Samy> sustrik, CAS is expensive.
[Wednesday 15 September 2010] [13:09:31] <Samy> sustrik, it’s best to avoid the lock prefix completely if possible.
[Wednesday 15 September 2010] [13:09:37] <sustrik> ack
[Wednesday 15 September 2010] [13:09:41] <Samy> sustrik, atomic loads and stores do this.
[Wednesday 15 September 2010] [13:10:08] <sustrik> atomic load & stores == have global ordering ?
[Wednesday 15 September 2010] [13:10:12] <Samy> sustrik, in theory, CAS can provide infinite consensus. If you don’t need infinite consensus, you might not need CAS at all.
[Wednesday 15 September 2010] [13:10:50] <sustrik> anyway, i have to get my head around it
[Wednesday 15 September 2010] [13:11:00] <sustrik> are you around somewhere?
[Wednesday 15 September 2010] [13:11:01] <Samy> Processor ordering.
[Wednesday 15 September 2010] [13:11:10] <sustrik> email, or so?
[Wednesday 15 September 2010] [13:11:29] <Samy> sustrik, I’d love your feedback on this library I’m working on sometime.
[Wednesday 15 September 2010] [13:11:36] <Samy> sustrik, what architectures does ZeroMQ support?
[Wednesday 15 September 2010] [13:12:26] <sustrik> we’ve tested on x86, itanium, sparc, ppc
[Wednesday 15 September 2010] [13:12:28] <sustrik> arm
[Wednesday 15 September 2010] [13:12:29] <Samy> sustrik, if you see section 8.2.2 in Volume 3 of the IA32 manuals, you’ll get a nice break-down of the memory ordering.
[Wednesday 15 September 2010] [13:12:58] <Samy> sustrik, does the ZeroMQ project have regular access to such boxes?
[Wednesday 15 September 2010] [13:13:21] <sustrik> we have an itanium and sparc box
[Wednesday 15 September 2010] [13:13:28] <sustrik> no ppc
[Wednesday 15 September 2010] [13:13:28] <Samy> Itanium, nice. :-)
[Wednesday 15 September 2010] [13:13:38] <Samy> My PPC box died (Mac Mini) recently. Need to fix it.
[Wednesday 15 September 2010] [13:14:01] <Samy> I sold my only decent SPARCv9 box, at work our SPARCv9 machines are tied to our build cluster.
[Wednesday 15 September 2010] [13:14:21] <Samy> sustrik, if I could get accounts on those, I can port my library’s atomic interface for them to support those.
[Wednesday 15 September 2010] [13:14:31] <Samy> sustrik, might be useful, at least as a reference implementation of some structures.
[Wednesday 15 September 2010] [13:15:09] <Samy> sustrik, ;]
[Wednesday 15 September 2010] [13:15:46] <sustrik> our sparc box is extremely old and slow :)
[Wednesday 15 September 2010] [13:15:54] <Samy> Itanium is fine too.
[Wednesday 15 September 2010] [13:16:07] <Samy> Well, Itanium is what I really want access to.
[Wednesday 15 September 2010] [13:16:10] <sustrik> but mato mumbled something about bringing some 14-core sparc
[Wednesday 15 September 2010] [13:16:16] <sustrik> ah
[Wednesday 15 September 2010] [13:16:36] <sustrik> i have to ask mato, i think there’s some applications running on it
[Wednesday 15 September 2010] [13:16:42] <keffo> sandy bridge ftw!
[Wednesday 15 September 2010] [13:16:48] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [13:17:20] <Samy> I look forward to future discussions. I would like to show you the work I’ve done so far for feedback.
[Wednesday 15 September 2010] [13:17:34] <Samy> It is C, not C++, however.
[Wednesday 15 September 2010] [13:17:41] <sustrik> Samy, i can have a look but as you see i am not an expert :)
[Wednesday 15 September 2010] [13:18:13] <Samy> Well, that’s fine. The idea is, what would it take to make you use this? As a 3rd party looking to integrate this into their product, what issues do you have with the interface?
[Wednesday 15 September 2010] [13:18:28] <Samy> etc
[Wednesday 15 September 2010] [13:18:35] <sustrik> what’s the link to the lib?
[Wednesday 15 September 2010] [13:18:43] <kleppari> we’re ditching a couple of sun t1000 boxes at work
[Wednesday 15 September 2010] [13:18:48] <Samy> No link. Tarball, sustrik.
[Wednesday 15 September 2010] [13:18:54] <Samy> When we have a discussion, I can share that.
[Wednesday 15 September 2010] [13:18:55] <kleppari> let me see if I can ‘rescue’ on of them
[Wednesday 15 September 2010] [13:19:07] <kleppari> s/on/one/
[Wednesday 15 September 2010] [13:19:19] Quit mrm2m has left this server (Quit: Leaving.).
[Wednesday 15 September 2010] [13:20:11] <Samy> Cool.
[Wednesday 15 September 2010] [13:20:16] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [13:20:19] <Samy> I’m back to work, take care.
[Wednesday 15 September 2010] [13:20:41] <sustrik> you too
[Wednesday 15 September 2010] [13:20:42] <sustrik> bye
[Wednesday 15 September 2010] [13:20:47] <sustrik> kleppari: i don’t think it’s really needed
[Wednesday 15 September 2010] [13:21:15] <sustrik> unless you need it yourself
[Wednesday 15 September 2010] [13:21:41] <kleppari> not really
[Wednesday 15 September 2010] [13:21:47] <kleppari> kind of lost interest in solaris after oracle bought it
[Wednesday 15 September 2010] [13:21:53] <kleppari> err, after oracle bought sun
[Wednesday 15 September 2010] [13:22:00] <kleppari> I think they’ll do a wonderful job killing it
[Wednesday 15 September 2010] [13:22:23] <sustrik> yeah, looks like there will be a lot of sun boxes avialable :)
[Wednesday 15 September 2010] [13:22:42] <kleppari> heh, yeah - red hat will make a fortune
[Wednesday 15 September 2010] [13:23:37] <kleppari> but the t1000 was a good box, I still think 16 concurrent threads of execution per core is impressive
[Wednesday 15 September 2010] [13:24:57] <kleppari> t2000, sorry
[Wednesday 15 September 2010] [13:27:14] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [13:29:59] <pieterh> kleppari: we used to have a t2000 at iMatix when we were making OpenAMQ
[Wednesday 15 September 2010] [13:30:11] <pieterh> it was a pretty impressive box, made a noise like an aircraft taking off
[Wednesday 15 September 2010] [13:30:30] <pieterh> and was slower (all 32 cores or whatever) than a 2-core Athlon
[Wednesday 15 September 2010] [13:31:10] Quit ModusPwnens has left this server.
[Wednesday 15 September 2010] [13:31:10] <mato> pieterh: it still might be more useful than the old e4500 i can get for the project
[Wednesday 15 September 2010] [13:31:16] <kleppari> slower at what?
[Wednesday 15 September 2010] [13:31:22] <mato> pieterh: 0mq is a very different codebase from OpenAMQ
[Wednesday 15 September 2010] [13:32:01] <pieterh> mato: perhaps, yes
[Wednesday 15 September 2010] [13:32:12] <mato> kleppari: if you do get a chance to rescue one i think it’d be more interesting for development of the lockfree algorithms in 0mq than what i’ve been offered
[Wednesday 15 September 2010] [13:32:14] <pieterh> kleppari: at raw I/O afaics
[Wednesday 15 September 2010] [13:32:24] <pieterh> but the main difference was probably Linux vs. Solaris
[Wednesday 15 September 2010] [13:32:25] <mato> kleppari: which is an old E4500 (fully spec’ed with 14 CPUs)
[Wednesday 15 September 2010] [13:32:34] <kleppari> I found that these boxes perform pretty well with a huge runqueue
[Wednesday 15 September 2010] [13:32:42] <kleppari> mato: let me see what I can do
[Wednesday 15 September 2010] [13:32:50] <pieterh> they were designed for web services, indeed
[Wednesday 15 September 2010] [13:32:56] <kleppari> mato: can’t promise anything, though, but I’ll try
[Wednesday 15 September 2010] [13:32:58] Quit xla has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [13:33:10] <mato> kleppari: no hurry; also, where are you located? it’s not worth shipping that stuff from outside of the EU, too expensive
[Wednesday 15 September 2010] [13:33:18] <kleppari> iceland
[Wednesday 15 September 2010] [13:33:22] <mato> lol :)
[Wednesday 15 September 2010] [13:33:29] <kleppari> so the shipping might kill the deal :P
[Wednesday 15 September 2010] [13:33:42] <mato> kleppari: precisely, unless you have friends at smyril line :-)
[Wednesday 15 September 2010] [13:33:54] <mato> “chuck it down with them bananas” :-)
[Wednesday 15 September 2010] [13:34:12] <kleppari> heheh
[Wednesday 15 September 2010] [13:35:50] <kleppari> I don’t think we’d be shipping bananas out of the country, but fair point :P
[Wednesday 15 September 2010] [13:36:15] <pieterh> hey... ebay is offering free shipping or something...
[Wednesday 15 September 2010] [13:36:54] <mato> kleppari: thanks for the offer in any case
[Wednesday 15 September 2010] [13:36:58] * mato is off to 0beer :-)
[Wednesday 15 September 2010] [13:37:05] <pieterh> 0beer0clock
[Wednesday 15 September 2010] [13:37:08] <mato> beer
[Wednesday 15 September 2010] [13:37:12] <pieterh> zbeer
[Wednesday 15 September 2010] [13:37:13] <mato> cyl
[Wednesday 15 September 2010] [13:37:18] <pieterh> cyrsn
[Wednesday 15 September 2010] [13:38:10] * pieterh suspects mato has a macro that types
[Wednesday 15 September 2010] [13:38:22] <mato> !
[Wednesday 15 September 2010] [13:38:56] <pieterh> :-) I knew it...
[Wednesday 15 September 2010] [13:39:08] <pieterh> oh... hang on...
[Wednesday 15 September 2010] [13:39:20] Quit AndrewBC has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [13:39:45] * pieterh wonders how to config xmodmap to do this...
[Wednesday 15 September 2010] [13:43:42] <kleppari> alt gr + o ?
[Wednesday 15 September 2010] [13:43:45] <kleppari> works on the .is layout
[Wednesday 15 September 2010] [13:44:07] <guido_g> and on 105 key .de (qwertz)
[Wednesday 15 September 2010] [13:44:39] <guido_g> and I just saw that ch. 3 is back...
[Wednesday 15 September 2010] [13:44:45] <pieterh> yeah, I fixed it
[Wednesday 15 September 2010] [13:45:03] * guido_g is reading
[Wednesday 15 September 2010] [13:45:14] Quit pieterh has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [13:46:48] <pieterh> cyal, i’m off for zbeer
[Wednesday 15 September 2010] [13:47:19] <pieterh> guido_g: there’s also a new problem solver in Ch1
[Wednesday 15 September 2010] [13:47:35] <guido_g> oh...
[Wednesday 15 September 2010] [13:50:32] <guido_g> nice one
[Wednesday 15 September 2010] [13:50:54] <guido_g> but can you do it in uml?
[Wednesday 15 September 2010] [13:51:01] <guido_g> ducks for cover
[Wednesday 15 September 2010] [13:52:53] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [13:53:25] <guido_g> more serious thing: smaller version that fits on one page so it can be printed easily
[Wednesday 15 September 2010] [14:04:28] <keffo> sustrik, What scenario could trigger a deadlock? (in single threaded proc.)
[Wednesday 15 September 2010] [14:59:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [15:09:33] <Samy> sustrik, ping?
[Wednesday 15 September 2010] [15:27:31] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [15:27:41] <ModusPwnens> Does zeromq have a lot of start up overhead of initialization?
[Wednesday 15 September 2010] [15:27:46] <ModusPwnens> or initialization*
[Wednesday 15 September 2010] [15:36:47] <ModusPwnens> the only reason I ask is because i keep getting strange results when I benchmark where the throughput is low intially, but then gradually increases
[Wednesday 15 September 2010] [15:47:34] Quit g5q has left this server (Quit: Bye~).
[Wednesday 15 September 2010] [15:51:53] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [15:54:03] Quit keffo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [16:19:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 15 September 2010] [16:36:23] Quit zirpu has left this server (Quit: kernel upgrade.).
[Wednesday 15 September 2010] [16:44:47] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [18:45:35] Quit sbahra has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [19:27:16] Quit danielqo has left this server (Quit: danielqo).
[Wednesday 15 September 2010] [19:56:19] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [20:24:19] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [21:17:56] Quit carrus85 has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [21:25:08] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [21:33:25] Quit andrewvc has left this server (Quit: andrewvc).
[Wednesday 15 September 2010] [21:39:21] Quit danielqo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [21:42:30] Quit danielqo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [21:48:19] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [21:52:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 15 September 2010] [22:17:24] <lestrrat> is there some sort of debug flag to enable tracing or some such for zeromq?
[Wednesday 15 September 2010] [22:17:46] <lestrrat> I’m having problems, and want to know what’s going on in the zeromq side...
[Wednesday 15 September 2010] [22:56:14] <dermoth> lestrrat, maybe strace would help, but it might be a bit low-level depending on what you need to figure out
[Wednesday 15 September 2010] [22:57:28] <dermoth> lestrrat, fwiw if you’re using short-lived processes then some queues have no way to guarantee all messages have been delivered before shutdown; when you call zmq_send data is only bufered in a local queue to be sent to the remote end
[Wednesday 15 September 2010] [23:03:52] <lestrrat> thanks, gonna have to RFE a TRACE variable or something ;)
[Thursday 16 September 2010] [00:09:44] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [01:06:23] Quit carrus85 has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [01:28:04] Quit guido_g has left this server (Quit: Boom...).
[Thursday 16 September 2010] [01:35:08] Quit carrus85 has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [01:36:14] Quit carrus85 has left this server (Client Quit).
[Thursday 16 September 2010] [01:52:22] Quit keffo has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [01:58:44] Quit
AndrewBC_ has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [02:03:48] Quit AndrewBC has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [03:02:56] Quit kleppari has left this server (Read error: Operation timed out).
[Thursday 16 September 2010] [03:12:21] <dermoth> I was trying out the php-zmq module today, and it appears some tests die with an assertion failed with –enable-pgm in the zeromq library
[Thursday 16 September 2010] [03:12:43] <dermoth> not that I need it though...
[Thursday 16 September 2010] [03:14:05] <dermoth> the error is: Assertion failed: !pgm_supported () (zmq.cpp:240)
[Thursday 16 September 2010] [03:17:20] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [03:33:05] Quit kenkeiter has left this server (Quit: kenkeiter).
[Thursday 16 September 2010] [03:34:16] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [03:47:30] <sustrik> dermoth: don’t ope 2 contexts
[Thursday 16 September 2010] [03:47:33] <sustrik> open
[Thursday 16 September 2010] [04:03:48] Nick Nickname already in use. Trying
travlr_.
[Thursday 16 September 2010] [04:03:49] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 16 September 2010] [04:03:50] CTCP Received Version request from frigg.
[Thursday 16 September 2010] [04:03:51] Topic The channel topic is “Welcome!”.
[Thursday 16 September 2010] [04:03:53] Quit travlr has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [04:03:57] Mode Channel modes: no messages from outside, topic protection
[Thursday 16 September 2010] [04:03:57] Created This channel was created on 2010-02-05 10:44.
[Thursday 16 September 2010] [04:09:59] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Thursday 16 September 2010] [04:22:28] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 16 September 2010] [04:23:19] Quit xla has left this server (Client Quit).
[Thursday 16 September 2010] [04:28:49] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 16 September 2010] [04:32:25] Nick Nickname already in use. Trying
travlr__.
[Thursday 16 September 2010] [04:32:26] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 16 September 2010] [04:32:27] CTCP Received Version request from frigg.
[Thursday 16 September 2010] [04:32:28] Topic The channel topic is “Welcome!”.
[Thursday 16 September 2010] [04:32:34] Mode Channel modes: no messages from outside, topic protection
[Thursday 16 September 2010] [04:32:34] Created This channel was created on 2010-02-05 10:44.
[Thursday 16 September 2010] [04:32:46] Quit
travlr_ has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [04:58:19] Quit xrfang has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [05:26:25] <lestrrat> cool, it wasn’t just me who got bit with multiple contexts + pgm ;)
[Thursday 16 September 2010] [06:05:00] <sustrik> lestrrat: steven mc coy have already started migrating 0mq to new version of openpgm that should have the problem solved
[Thursday 16 September 2010] [06:05:19] <sustrik> but it’s work in progress, no guarantees
[Thursday 16 September 2010] [07:18:29] Quit jsimmons has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [08:03:25] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [09:05:38] <psino> in zmq 2.1, zmq_close wont discard that is queued for sending – is anything similar going to be implemented for data that has arrived, and are pending recv-calls?
[Thursday 16 September 2010] [09:21:00] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [09:30:00] Quit guido_g has left this server (Quit: Boom...).
[Thursday 16 September 2010] [09:44:05] <joyenlee> Hi!
[Thursday 16 September 2010] [09:44:52] Quit joyenlee has left this server (Quit: Page closed).
[Thursday 16 September 2010] [09:49:06] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 16 September 2010] [10:00:22] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [10:02:55] Quit iFire has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [10:30:31] Quit mato has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [10:35:09] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [10:37:42] Quit grahamalot has left this server (Read error: Operation timed out).
[Thursday 16 September 2010] [11:33:31] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [11:35:38] Quit psino has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [11:56:30] Quit psino has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [12:45:29] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [13:48:38] Quit AndrewBC has left this server (Ping timeout: 264 seconds).
[Thursday 16 September 2010] [14:02:55] Quit xla has left this server (Quit: leaving).
[Thursday 16 September 2010] [14:14:38] Quit psino has left this server (Quit: psino).
[Thursday 16 September 2010] [14:17:11] Quit keffo has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [14:17:17] Quit ModusPwnens has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [14:32:59] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [14:45:08] Quit AndrewBC has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [15:07:01] Join yrashk has joined this channel (
~yrashk@70.28.245.3).
[Thursday 16 September 2010] [15:09:10] Quit fredix has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [15:09:22] Quit omarkj has left this server (Quit: omarkj).
[Thursday 16 September 2010] [15:14:22] Join fredix has joined this channel (
~fredix@2a01:e35:2f38:d790:218:2dff:fe00:173).
[Thursday 16 September 2010] [15:56:08] Quit yrashk has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [17:15:50] <dermoth> sustrik, i didn’t write the test... but why would it fail? What is multiple applications use different context on the same server? do you think the contect should be abstracted in the PHP module?
[Thursday 16 September 2010] [17:43:05] Quit g5q has left this server (Quit: Bye~).
[Thursday 16 September 2010] [18:12:59] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Thursday 16 September 2010] [18:13:23] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [18:36:54] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [18:42:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [18:46:48] Quit keffo has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [19:13:13] Quit andrewvc has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [19:21:52] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [20:11:41] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 16 September 2010] [20:15:04] Quit jhawk28 has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [20:45:07] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Thursday 16 September 2010] [20:46:51] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [20:57:43] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [21:01:31] Quit plq has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [21:23:48] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [21:24:04] Quit jhawk28 has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [21:31:14] Quit erickt has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [21:41:14] <
mattp_> Hi. I’m trying to build using Cygwin. I’m getting me an error message telling me to install e2fsprog even though I’ve already got it.
[Thursday 16 September 2010] [21:41:18] <
mattp_> Any suggestions?
[Thursday 16 September 2010] [21:42:22] <
mattp_> (not sure how that “me” got into my third sentence. :) )
[Thursday 16 September 2010] [21:49:23] <
mattp_> crickets...
[Thursday 16 September 2010] [21:50:38] Quit
mattp_ has left this server (Quit: Using leafChat 2).
[Thursday 16 September 2010] [21:51:44] Quit
mattp_ has left this server (Client Quit).
[Thursday 16 September 2010] [21:52:32] <mpassell> thought I’d try through a different client. Anyone out there?
[Thursday 16 September 2010] [22:01:10] Part mpassell has left this channel.
[Thursday 16 September 2010] [22:12:57] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [23:28:16] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Thursday 16 September 2010] [23:51:02] <chowmeined> can one multiplex different message patterns on a single socket?
[Friday 17 September 2010] [00:18:42] <andrewvc> cremes, by chance are you around?
[Friday 17 September 2010] [00:19:25] Part snerd has left this channel.
[Friday 17 September 2010] [00:33:45] <dermoth> iirc there used to be device config example, looks like they’re gone. why?
[Friday 17 September 2010] [00:33:56] <dermoth> king of easier than reading the code :)
[Friday 17 September 2010] [00:38:41] <guido_g> maybe thats what you want
[Friday 17 September 2010] [00:40:27] <dermoth> guido_g, thanks, looks nice. Actually for now i’m just using the stock devices that come with zeromq. The config quite basic, in xml. I was justt wondering where they gone
[Friday 17 September 2010] [00:40:36] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [00:41:11] <guido_g> afair there going to be moved out of the core distribution
[Friday 17 September 2010] [00:42:44] <dermoth> ok... well I kind of like them for the most basic usage... I could replace the config with command-line arguments though, imho it would be much easier
[Friday 17 September 2010] [00:43:05] <guido_g> dermoth: btw, please send your patch to the ml with a statement that it is licensed under mit/x11, othwerwise it can’t be accepted
[Friday 17 September 2010] [00:43:33] <dermoth> really?
[Friday 17 September 2010] [00:44:25] <guido_g> it’s the procedure
[Friday 17 September 2010] [00:44:50] <guido_g> has been discussed here and on the ml
[Friday 17 September 2010] [00:46:13] <guido_g> dermoth: easiest way is to use format-patch and append that to the mal w/ the license statement
[Friday 17 September 2010] [00:46:15] <dermoth> then how about I sen you this?
[Friday 17 September 2010] [00:46:19] <dermoth> perl -pi -e ‘s/ZMQ_DOWNSTREAM/ZMQ_PUSH/; s/ZMQ_UPSTREAM/ZMQ_PULL/;’ devices/zmq_streamer/zmq_streamer.cpp
[Friday 17 September 2010] [00:46:20] <guido_g> not me
[Friday 17 September 2010] [00:46:36] <guido_g> dermoth: see git format-patch
[Friday 17 September 2010] [00:46:50] <dermoth> git commit -m’zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL’
[Friday 17 September 2010] [00:47:12] <dermoth> I use git-format-patch... but I’m not even subscrived to the ml
[Friday 17 September 2010] [00:47:29] <guido_g> too bad
[Friday 17 September 2010] [00:47:35] <dermoth> the two commands abot will get you the same result less the credit
[Friday 17 September 2010] [00:48:02] <guido_g> what credit?
[Friday 17 September 2010] [00:48:11] <dermoth> the author in commit message
[Friday 17 September 2010] [00:48:33] <guido_g> ah
[Friday 17 September 2010] [00:48:55] <dermoth> unless you’re authorised to do: git commit -m’zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL’ ‘–author=Thomas Guyot-Sionnest <
thomas@blinkx.com>’
[Friday 17 September 2010] [00:48:56] <guido_g> i’m more concerned about the license statement, w/o it the patch can’t go in
[Friday 17 September 2010] [00:49:40] <dermoth> it’s not a patch it a command that patsh, there shouldn’t be any legal issue. if you run the command you have the copyright for the resulting code ;)
[Friday 17 September 2010] [00:50:13] <guido_g> i think that is clear
[Friday 17 September 2010] [00:50:50] <guido_g> i just stated what’s needed to get a patch in, because a pull-req is a patch
[Friday 17 September 2010] [00:54:01] Quit kenkeiter has left this server (Quit: kenkeiter).
[Friday 17 September 2010] [00:54:07] <dermoth> I understand the legal basis, but I will likely not send a patch that takes me more time to “send” thna write... Imagine trying that with the Linux Kernel!
[Friday 17 September 2010] [00:54:48] <guido_g> as i said, the device apps will be moved out of the core, afaik
[Friday 17 September 2010] [00:55:01] <guido_g> so...
[Friday 17 September 2010] [00:55:12] <dermoth> so I guess there won’t be any trouble getting that in, whatever method you use... on that i’ll get back to my testing :)
[Friday 17 September 2010] [00:55:25] <guido_g> dermoth: wrong
[Friday 17 September 2010] [00:55:35] <guido_g> the license issue is a problem atm
[Friday 17 September 2010] [00:55:41] <guido_g> ask pieter about it
[Friday 17 September 2010] [00:56:17] <guido_g> and go read the ml (via gmane w/o registration)
[Friday 17 September 2010] [00:56:51] <dermoth> what I meant is that it probably take you less time to rename it yourself than using my patch (and I use gmane...)
[Friday 17 September 2010] [00:57:57] <dermoth> i wouldn’t mind taking the time to post on the ml for a real change... and can’T I put it in the commit message too?
[Friday 17 September 2010] [00:58:14] <guido_g> cool, then you can easily find the threads about patches and legal issues
[Friday 17 September 2010] [00:58:43] <guido_g> if it were in the commit message, even better
[Friday 17 September 2010] [00:58:44] <dermoth> except that i’d rather catch some much needed sleep
[Friday 17 September 2010] [00:58:48] <dermoth> ok
[Friday 17 September 2010] [01:04:50] <dermoth> here you go... next time i’ll know :)
[Friday 17 September 2010] [01:12:46] <guido_g> hey, i didn’t invent the laws and such, just repeating the handbook
[Friday 17 September 2010] [01:13:07] <dermoth> Any PECL developers here? IMHO the PHP api should have a cb that is called on persistent socket creation, so we could create the socket and connect in one shot
[Friday 17 September 2010] [01:13:24] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [01:33:13] Quit carrus85 has left this server (Quit: Leaving).
[Friday 17 September 2010] [01:33:57] Join kenkeiter has joined this channel (
~kenkeiter@2002:4c73:685:1234:223:6cff:fe98:be3b).
[Friday 17 September 2010] [01:36:54] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [01:37:00] Quit kenkeiter has left this server (Client Quit).
[Friday 17 September 2010] [01:40:32] <sleeperbot> this driver takes up a ton of CPU
[Friday 17 September 2010] [01:42:32] <sleeperbot> the way that it is implemented =/
[Friday 17 September 2010] [01:48:29] <yrashk> sleeperbot afair there’s none
[Friday 17 September 2010] [01:48:41] <yrashk> but supposedly I will need one soon-ish
[Friday 17 September 2010] [01:48:54] <yrashk> so might invest some time in getting it done properly
[Friday 17 September 2010] [01:51:51] <sleeperbot> i see, thanks for the info
[Friday 17 September 2010] [01:52:55] <yrashk> yw
[Friday 17 September 2010] [01:54:32] Quit keffo has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [02:20:26] Quit sleeperbot has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [02:44:10] Quit jsimmons has left this server (Client Quit).
[Friday 17 September 2010] [02:52:57] Quit xrfang has left this server (Remote host closed the connection).
[Friday 17 September 2010] [03:02:27] <sustrik> psino: how would it work for recv calls?
[Friday 17 September 2010] [03:03:38] <sustrik> dermoth: it’s a problem with OpenPGM, which will be solved soon; what i was saying is that in most cases you have just one context so you never encounter the problem
[Friday 17 September 2010] [03:04:38] <sustrik> chowmeined: no, you can’t
[Friday 17 September 2010] [03:04:59] Quit kleppari has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [03:06:05] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Friday 17 September 2010] [03:09:17] Quit guido_g has left this server (Quit: Boom...).
[Friday 17 September 2010] [03:21:08] Quit fredix has left this server (Read error: Operation timed out).
*** on Fri Sep 17 04:13:00 2010
[Friday 17 September 2010] [04:13:00] Topic The channel topic is “Welcome!”.
[Friday 17 September 2010] [04:13:04] Mode Channel modes: no messages from outside, topic protection
[Friday 17 September 2010] [04:13:04] Created This channel was created on 2010-02-05 10:44.
[Friday 17 September 2010] [04:49:41] <ivan-korobkov> Hi, I use zmq 2.0.9 + epgm + the latest python bindings. Sometimes it crashes with Assertion failed:
*tmpbuf > 0 (zmq_decoder.cpp:60). Does anybody know why it happens? Is it a bug and I should create an issue?
[Friday 17 September 2010] [05:05:08] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 17 September 2010] [05:08:09] <guido_g> try to isolate it in a small test case, so others can verify it
[Friday 17 September 2010] [05:12:03] <ivan-korobkov> OK, I will try, but it happens rather sporadically.
[Friday 17 September 2010] [05:13:15] <guido_g> will be no problem, if we have a program that can run
[Friday 17 September 2010] [05:17:15] Quit xrfang has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [05:31:16] <psino> sustrik: In the case of a pull socket, I was thinking it could disconnect from the push socket and still be able to recv() on the messages the push socket has sent to it
[Friday 17 September 2010] [05:40:43] <sustrik> ivan-korobkov: create an issue
[Friday 17 September 2010] [05:41:15] <sustrik> psino: any idea of how the API should look like?
[Friday 17 September 2010] [05:46:23] <psino> sustrik: from the end users point of view, much like now, except that after calling socket.close(), a poller wouldn’t put the socket in exceptional state before the messages that has been sent to it from the other side has been recv()’ed
[Friday 17 September 2010] [05:49:07] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [05:56:48] Quit mato has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:10:01] Quit ivan-korobkov has left this server (Quit: Leaving).
[Friday 17 September 2010] [06:15:51] Quit guido_g has left this server (Quit: Boom...).
[Friday 17 September 2010] [06:16:50] Quit iFire has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:28:18] Quit
AndrewBC_ has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:28:19] Quit plq has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:32:59] <CIA-20> zeromq2: 03Martin Sustrik 07master * r85aa25e 10/ src/req.cpp : bug in REQ+multipart fixed -
http://bit.ly/bzWD3V[Friday 17 September 2010] [06:39:48] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:46:52] Quit omarkj has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:46:53] Nick
omarkj_ is now known as omarkj.
[Friday 17 September 2010] [06:47:41] Quit pieterh has left this server (Quit: Leaving).
[Friday 17 September 2010] [06:50:32] Quit mato has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [06:58:32] <CIA-20> zeromq2: 03Martin Sustrik 07master * rc7b9ba3 10/ (src/command.hpp src/session.cpp): reconnection process fixed when failure occurs during init phase -
http://bit.ly/cIBUcl[Friday 17 September 2010] [06:58:58] Quit
AndrewBC_ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [08:10:38] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [08:50:47] Quit sbahra has left this server (Remote host closed the connection).
[Friday 17 September 2010] [10:05:01] Join plq has joined this channel (
~plq@78.189.30.218).
[Friday 17 September 2010] [10:09:28] Quit bgranger has left this server (Quit: bgranger).
[Friday 17 September 2010] [10:27:57] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 17 September 2010] [10:32:15] Quit plq has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [12:28:09] Quit cremes has left this server (Quit: cremes).
[Friday 17 September 2010] [12:28:48] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Friday 17 September 2010] [12:34:43] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [12:36:18] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [13:29:54] Quit yrashk has left this server (Ping timeout: 246 seconds).
[Friday 17 September 2010] [13:39:04] Quit omarkj has left this server (Quit: omarkj).
[Friday 17 September 2010] [14:48:58] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [14:51:00] Quit __moore__ has left this server (Remote host closed the connection).
[Friday 17 September 2010] [14:55:33] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [15:00:22] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [15:05:32] Quit kelvink has left this server (Quit: Leaving.).
[Friday 17 September 2010] [16:02:02] <bgranger> sustrik: question about msg ref counting...
[Friday 17 September 2010] [16:11:22] <keffo> I think they’re all off work :)
[Friday 17 September 2010] [16:11:32] <keffo> starting to think I’m the only one around anymore :)
[Friday 17 September 2010] [16:12:29] <keffo> lol, last time anyone even spoke in here was about 11h ago :)
[Friday 17 September 2010] [16:12:39] <andrewvc> and you just broke the record
[Friday 17 September 2010] [16:12:55] <keffo> no, bgranger did 10 mins ago :)
[Friday 17 September 2010] [16:14:09] <andrewvc> oh damn, hehe, that’s what I get for sleeping my laptop, missing all the action
[Friday 17 September 2010] [16:16:43] <keffo> :)
[Friday 17 September 2010] [16:17:16] <keffo> Does anyone know best-practices for transferring large amounts of data? Simply to split into multipart?
[Friday 17 September 2010] [16:17:19] Quit yrashk has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [16:17:46] <andrewvc> I remember there were benchmarks on the zeromq site with various message sizes, ranging up to something huge
[Friday 17 September 2010] [16:18:22] <keffo> right, but simply sending one huge message just sounds like mem hog to me
[Friday 17 September 2010] [16:29:58] Quit xla has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [16:32:34] <sustrik> i am back
[Friday 17 September 2010] [16:32:49] <sustrik> bgranger: what’s the question?
[Friday 17 September 2010] [16:34:24] <sustrik> keffo: so is multipart message, you have to break it into sequence of messages
[Friday 17 September 2010] [16:34:42] <sustrik> then you have to define your own transfer protocol etc.
[Friday 17 September 2010] [16:34:45] <sustrik> it’s not easy
[Friday 17 September 2010] [16:34:51] <keffo> mm
[Friday 17 September 2010] [16:35:01] <sustrik> what do you want to do?
[Friday 17 September 2010] [16:35:31] <keffo> To be honest I’m leaning towards an accompanying torrent lib for larger transfers
[Friday 17 September 2010] [16:35:46] <keffo> I solved both my deadlock and garbled data btw..
[Friday 17 September 2010] [16:35:59] <keffo> as expected it was all me :)
[Friday 17 September 2010] [16:36:03] <keffo> CRT mismatch actually
[Friday 17 September 2010] [16:36:06] <sustrik> :)
[Friday 17 September 2010] [16:36:34] <sustrik> yeah, makes sense, you would have to implement torrent kind of behaiour yourself on top of 0mq
[Friday 17 September 2010] [16:36:55] <bgranger> sustrik: we have a very subtle question about ref counting of msgs...
[Friday 17 September 2010] [16:37:16] <keffo> sustrik, I was thinking more along the lines of using the transmission codebase, or libtorrent..
[Friday 17 September 2010] [16:37:57] <sustrik> bgranger: yes?
[Friday 17 September 2010] [16:38:00] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [16:38:10] <bgranger> Hi
[Friday 17 September 2010] [16:38:11] <sustrik> keffo: sure
[Friday 17 September 2010] [16:38:14] <sustrik> hi
[Friday 17 September 2010] [16:38:22] <bgranger> We are trying to understand the refcounting
[Friday 17 September 2010] [16:38:40] <bgranger> We see that zmq_msg_close is the place where the ffn is called if the ref count is 0
[Friday 17 September 2010] [16:39:10] <bgranger> What exactly does zmq_send do with the ref count. What is zmq_close is called before a msg is actually send?
[Friday 17 September 2010] [16:39:26] <bgranger> Does send somehow end up calling zmq_msg_close?
[Friday 17 September 2010] [16:42:42] <sustrik> yes, send takes ownership of your message
[Friday 17 September 2010] [16:43:03] <bgranger> OK, we just saw that at the end of xsend there is
[Friday 17 September 2010] [16:43:07] <sustrik> your message object will be empty after return from the send call
[Friday 17 September 2010] [16:43:10] <bgranger> int rc = zmq_msg_init (
msg_);
[Friday 17 September 2010] [16:43:30] <sustrik> yes, that initialises your message to empty
[Friday 17 September 2010] [16:43:31] <bgranger> So if you then call zmq_msg_close you are not closing the original msg, but a new empty one.
[Friday 17 September 2010] [16:43:38] <bgranger> Great!
[Friday 17 September 2010] [16:43:39] <sustrik> exactly
[Friday 17 September 2010] [16:43:59] <bgranger> We are getting python to know when all copies of a message are done being sent.
[Friday 17 September 2010] [16:44:29] <sustrik> intercepting the free function?
[Friday 17 September 2010] [16:44:41] <bgranger> There is a subtle issue with non-copy sends. The user has to be able to know when zmq is done with it and they can edit the buffer again.
[Friday 17 September 2010] [16:44:46] <bgranger> Yes, we use the ffn.
[Friday 17 September 2010] [16:45:13] <bgranger> Otherwise the user might accidentally start the edit the buffer they sent before zmq has send it...
[Friday 17 September 2010] [16:45:29] <bgranger> non-copy sends are super subtle...
[Friday 17 September 2010] [16:45:34] <sustrik> so you are basically writing your own allocator
[Friday 17 September 2010] [16:46:21] <bgranger> We use Pythons garbage collection mechanism along with our own ref counts that allow us to track the underlying zmq ref counts.
[Friday 17 September 2010] [16:46:45] <sustrik> what’s the goal?
[Friday 17 September 2010] [16:47:12] <bgranger> In Python you might have a mutable array of numbers that you want to send using non-copy.
[Friday 17 September 2010] [16:47:27] <bgranger> You send it, and maybe then you want to modify it.
[Friday 17 September 2010] [16:47:37] <sustrik> i see
[Friday 17 September 2010] [16:47:40] <bgranger> But if you modify it before zmq sends it, the wrong thing gets sent
[Friday 17 September 2010] [16:47:54] <bgranger> This is common in Python with numerical arrays (matrices, etc.)
[Friday 17 September 2010] [16:48:23] <bgranger> For large static data it is not a problem, but in numerical/scientific computing, most data structures are mutable...
[Friday 17 September 2010] [16:49:08] <bgranger> sustrik: I think we understand the refcounting better now though. Thanks!
[Friday 17 September 2010] [16:50:55] <sustrik> np
[Friday 17 September 2010] [16:51:04] <sustrik> a question though:
[Friday 17 September 2010] [16:51:33] <sustrik> doesn’t it turn the messaging into sync model?
[Friday 17 September 2010] [16:51:43] <sustrik> one-message-a-time?
[Friday 17 September 2010] [17:04:34] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [17:08:59] <bgranger> sustrik: back from lunch...
[Friday 17 September 2010] [17:09:15] <bgranger> no the messaging is still fully async
[Friday 17 September 2010] [17:09:21] <minrk> sustrik: I’m not sure what your question means. All we did was add an object that tracks whether a message no longer has references in zmq
[Friday 17 September 2010] [17:09:48] <bgranger> You can then query that object to learn if the message has fully been sent.
[Friday 17 September 2010] [17:10:03] <bgranger> send does not block any more than it used to
[Friday 17 September 2010] [17:10:29] <minrk> or if there are any message objects pointing to it at all
[Friday 17 September 2010] [17:11:15] <bgranger> minrk: it would be nice if we could have a method of MessageTracker that would block until the msg is not pending though.
[Friday 17 September 2010] [17:12:56] <minrk> that’s easy
[Friday 17 September 2010] [17:13:34] Quit keffo has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [17:13:42] <bgranger> minrk: I am ready when you are....just give me a call on skype
[Friday 17 September 2010] [17:13:50] <minrk> okay
[Friday 17 September 2010] [17:14:08] <minrk> give me a few minutes
[Friday 17 September 2010] [17:15:27] <bgranger> no problem...
[Friday 17 September 2010] [17:20:09] <minrk> okay
[Friday 17 September 2010] [17:22:24] <mrossi> anyone know who is on the sending side of the signaler instance inside of the app_thread class?
[Friday 17 September 2010] [17:24:07] <mrossi> have an app that occasionally gets hung at line 88 of app_thread.cpp doing a signaler.recv()
[Friday 17 September 2010] [17:38:05] <dbudworth> question from a newbie zeromq user. I’d like to implement a socket server type system where the server is stateful for each client connection. Is there a zeromq socket type for this? it appears as though the server doesn’t have any context about which client it’s sending to at any one time
[Friday 17 September 2010] [17:46:16] Quit mrossi has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [18:09:29] Quit gavinstark has left this server (Remote host closed the connection).
[Friday 17 September 2010] [18:17:04] Quit
miguelito__ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [18:18:02] Quit skaar has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [18:34:58] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [18:35:02] Part minrk has left this channel.
[Friday 17 September 2010] [18:36:53] Quit dbudworth has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [18:51:56] <ModusPwnens> Hmm, can anyone take a look at some results from benchmarking tests?
[Friday 17 September 2010] [18:52:16] <ModusPwnens> I am seeing some strange results, but maybe they are normal
[Friday 17 September 2010] [19:25:30] Quit bgranger has left this server (Read error: Operation timed out).
[Friday 17 September 2010] [19:40:28] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Friday 17 September 2010] [19:48:48] Quit rossij has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [20:05:09] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [20:46:37] Quit jhawk28 has left this server (Remote host closed the connection).
[Friday 17 September 2010] [20:50:24] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [20:52:20] Quit bennymack has left this server (Write error: Broken pipe).
[Friday 17 September 2010] [21:15:43] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Friday 17 September 2010] [21:27:38] Quit ModusPwnens has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [21:50:26] Quit yrashk has left this server (Ping timeout: 264 seconds).
[Friday 17 September 2010] [22:10:08] Quit chowmeined has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [22:14:32] Quit chowmeined has left this server (Client Quit).
[Friday 17 September 2010] [23:01:11] Quit omarkj has left this server (Quit: omarkj).
[Friday 17 September 2010] [23:08:31] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:25:41] Quit
aleator_ has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [23:27:02] Quit bgranger has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:27:02] Quit bennymac1 has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:27:02] Quit Tasser has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:38:47] <benford> good morning folks. can anyone answer a question about non blocking recieve and send?
[Friday 17 September 2010] [23:45:02] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [00:02:58] Quit beppu has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [00:03:18] Quit skaar has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [00:19:11] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Saturday 18 September 2010] [01:42:31] Quit yrashk has left this server (Ping timeout: 255 seconds).
[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0c215fa 10/ (src/fq.cpp src/lb.cpp src/socket_base.cpp): potential duplicate termination of pipes removed -
http://bit.ly/9s1gJt[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rfb6ce53 10/ src/own.cpp : more details on launch_sibling in comments -
http://bit.ly/bQt0hE[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r4c6d07d 10/ (15 files): single term ack counting mechanism for every socket (no separate mechanisms for fq_t and lb_t) -
http://bit.ly/cS50VY[Saturday 18 September 2010] [02:17:24] <sustrik> benford: shoot
[Saturday 18 September 2010] [02:17:33] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Saturday 18 September 2010] [02:18:44] <benford> hi, i think it was user error :-) i’m trying to wrap the concurrent eventlet library around socket recv/send and was having some tryouble with polling
[Saturday 18 September 2010] [02:20:03] <sustrik> ok
[Saturday 18 September 2010] [02:29:55] <benford> actually - i guess i’ve got another question: if i’m doing a zmq_poll what might throw errno 14 ‘Bad address’
[Saturday 18 September 2010] [02:30:36] <benford> i’m probably calling this with an int representing an eventlet socket so i’m thinking that the flags are causing some sort of error?
[Saturday 18 September 2010] [02:30:57] <benford> (this is in pyzmq btw)
[Saturday 18 September 2010] [02:34:21] Quit bgranger has left this server (Quit: bgranger).
[Saturday 18 September 2010] [02:34:37] <benford> when using zmq_poll do i still have to stick with the POLLIN and POLLOUT flags or would i need to replace them with the equvalent select flags?
[Saturday 18 September 2010] [02:37:34] <benford> doh - figured it - was using a different context in the same thread
[Saturday 18 September 2010] [02:45:29] Quit mikko has left this server (Remote host closed the connection).
[Saturday 18 September 2010] [03:02:32] Quit kleppari has left this server (Read error: Operation timed out).
[Saturday 18 September 2010] [03:11:13] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Saturday 18 September 2010] [05:05:21] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [05:39:22] Quit kelvink has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [05:51:08] Quit yrashk has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [05:55:50] Join yrashk has joined this channel (
~yrashk@70.28.245.1).
[Saturday 18 September 2010] [06:05:08] Quit benford has left this server (Quit: benford).
[Saturday 18 September 2010] [06:08:04] Quit xla has left this server (Ping timeout: 272 seconds).
[Saturday 18 September 2010] [06:24:34] Quit mitkok has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [06:41:30] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [06:42:54] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Saturday 18 September 2010] [06:43:58] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [07:40:04] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [08:25:40] Quit xla has left this server (Quit: leaving).
[Saturday 18 September 2010] [09:06:58] Quit plq has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [09:20:20] Quit keffo has left this server (Quit: Leaving).
[Saturday 18 September 2010] [09:21:27] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [10:11:13] Quit benford has left this server (Client Quit).
[Saturday 18 September 2010] [10:14:33] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 18 September 2010] [10:17:41] Quit psino has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [10:45:41] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [11:25:25] Part mr19 has left this channel.
[Saturday 18 September 2010] [11:29:48] Quit mitkok has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [12:08:35] Quit plq has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [12:12:08] Quit bgranger has left this server (Quit: bgranger).
[Saturday 18 September 2010] [12:20:41] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [12:37:13] Quit rossij has left this server (Ping timeout: 245 seconds).
[Saturday 18 September 2010] [13:08:53] Quit xla has left this server (Ping timeout: 245 seconds).
[Saturday 18 September 2010] [13:15:58] Quit benford has left this server (Quit: benford).
[Saturday 18 September 2010] [13:52:50] Quit benford has left this server (Client Quit).
[Saturday 18 September 2010] [14:16:48] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [15:24:28] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [15:28:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [15:30:08] Quit ModusPwnens has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [15:32:21] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [16:01:51] <mr19> docs say to call zmq_init() from the “start of your main line of code”, my app starts a new thread that calls zmq_init() (via the zmq::context_t constructor) at thread startup. can easily be switched but curious as to whether this can cause problems. quick look through the code didn’t raise any flags to me.
[Saturday 18 September 2010] [16:06:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Saturday 18 September 2010] [16:34:20] <sustrik> mr19: to be able to communicate in inter-thread manner you have t share the context
[Saturday 18 September 2010] [16:34:42] <sustrik> i mean inproc:// transport
[Saturday 18 September 2010] [17:15:25] Quit sbahra has left this server (Ping timeout: 252 seconds).
[Saturday 18 September 2010] [17:27:14] <mr19> ok, i’m just using tcp and only from within that thread, just trying to simplify my code – i’m going to move the ctx instantiation to the main thread but was just curious if there are any other reasons for requiring the call to zmq_init() in the main thread
[Saturday 18 September 2010] [17:33:01] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [18:23:34] Quit sbahra has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [18:28:01] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [18:33:34] Quit psino has left this server (Client Quit).
[Saturday 18 September 2010] [19:44:12] Quit kelvink has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [19:47:08] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [20:11:50] Quit bbigras has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [21:00:12] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Saturday 18 September 2010] [22:16:16] Part mr19 has left this channel.
[Saturday 18 September 2010] [23:35:34] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Saturday 18 September 2010] [23:57:08] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [00:41:43] <dbudworth> is there a socket type that’s usable for request/streaming response type communications? ie: N clients open connection to a server and the server has a unique conversation with each client. (similar to regular socket accept/listen model)
[Sunday 19 September 2010] [01:09:28] <sustrik> if you want to control each socket, you need a regular socket
[Sunday 19 September 2010] [02:28:26] Quit bgranger has left this server (Quit: bgranger).
[Sunday 19 September 2010] [03:04:46] Quit kleppari has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [03:06:08] <CIA-20> zeromq2: 03Martin Sustrik 07master * r5153b63 10/ (src/session.cpp src/session.hpp): obsolete ‘active’ flag removed from session_t -
http://bit.ly/cRgwVE[Sunday 19 September 2010] [03:06:08] <CIA-20> zeromq2: 03Martin Sustrik 07master * r1f10208 10/ src/session.cpp : termination of pipe via delimiter message could stuck when no data were read from the pipe (because connection wasn’t active) – fixed -
http://bit.ly/dpganD[Sunday 19 September 2010] [03:06:18] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Sunday 19 September 2010] [03:12:19] Quit ikolomiets has left this server (Ping timeout: 252 seconds).
[Sunday 19 September 2010] [04:18:53] Quit mitkok has left this server (Ping timeout: 245 seconds).
[Sunday 19 September 2010] [04:51:01] Quit AndrewBC has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [05:53:38] Quit psino has left this server (Changing host).
[Sunday 19 September 2010] [05:53:41] Quit guido_g has left this server (Excess Flood).
[Sunday 19 September 2010] [06:43:04] Quit psino has left this server (Quit: psino).
[Sunday 19 September 2010] [08:00:05] Quit keffo has left this server (Ping timeout: 255 seconds).
[Sunday 19 September 2010] [09:02:38] Quit plq has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [09:22:03] Quit xla has left this server (Quit: leaving).
[Sunday 19 September 2010] [09:28:25] Quit iFire has left this server (Remote host closed the connection).
[Sunday 19 September 2010] [10:49:40] Quit psino has left this server (Quit: psino).
[Sunday 19 September 2010] [11:14:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 19 September 2010] [12:47:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 19 September 2010] [12:58:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 19 September 2010] [13:12:36] Quit keffo has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [14:02:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [14:27:33] Part sigmonsays has left this channel (“Leaving”).
[Sunday 19 September 2010] [15:15:03] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [15:21:20] Quit keffo has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [15:46:03] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [15:59:58] <CIA-20> zeromq2: 03Martin Sustrik 07master * r471013a 10/ src/zmq_init.cpp : two races fixed in session creation -
http://bit.ly/9UgZMy[Sunday 19 September 2010] [15:59:59] <CIA-20> zeromq2: 03Martin Sustrik 07master * r938689a 10/ (src/ctx.cpp src/zmq_init.cpp): synchronisation problem fixed in ctx_t -
http://bit.ly/bcX8OW[Sunday 19 September 2010] [16:04:03] Part AndrewBC has left this channel (“Leaving”).
[Sunday 19 September 2010] [16:21:47] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 19 September 2010] [16:25:35] Quit mitkok has left this server (Quit: Leaving.).
[Sunday 19 September 2010] [16:27:56] Quit rgl has left this server (Quit: Leaving).
[Sunday 19 September 2010] [16:50:14] <CIA-20> zeromq2: 03Martin Sustrik 07master * rcda3c96 10/ src/ctx.cpp : minor code reorganisation in ctx_t::terminate -
http://bit.ly/96qYvk[Sunday 19 September 2010] [16:50:16] <CIA-20> zeromq2: 03Martin Sustrik 07master * r1d23997 10/ (src/zmq_engine.cpp src/zmq_engine.hpp src/zmq_init.cpp): zmq_init_t destroyed zmq_engine_t before plugging it out from the poller first – fixed -
http://bit.ly/cvBySe[Sunday 19 September 2010] [16:50:38] Quit psino has left this server (Ping timeout: 255 seconds).
[Sunday 19 September 2010] [16:50:38] Nick
psino_ is now known as psino.
[Sunday 19 September 2010] [17:26:49] Quit plq has left this server (Remote host closed the connection).
[Sunday 19 September 2010] [17:58:28] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Sunday 19 September 2010] [18:06:05] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [18:06:23] Quit mitkok has left this server (Client Quit).
[Sunday 19 September 2010] [18:07:37] <CIA-20> zeromq2: 03Martin Sustrik 07master * r328c92a 10/ (4 files): problem with engine being attached to session while it’s being terminated fixed -
http://bit.ly/aVsix0[Sunday 19 September 2010] [19:15:32] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [20:01:47] Quit carrus85 has left this server (Quit: Leaving).
[Sunday 19 September 2010] [20:07:30] Quit
miguelito_ has left this server (Ping timeout: 252 seconds).
[Sunday 19 September 2010] [20:39:59] Quit miguelito has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [23:06:30] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Sunday 19 September 2010] [23:20:05] <stoneman> hello
[Sunday 19 September 2010] [23:20:10] Join dirtmcgirt has joined this channel (
~dirtmcgir@2002:62f8:7ff4:0:daa2:5eff:fe98:8f1c).
[Sunday 19 September 2010] [23:20:30] <stoneman> is anyone here?
[Sunday 19 September 2010] [23:21:36] Quit dirtmcgirt has left this server (Client Quit).
[Sunday 19 September 2010] [23:27:17] <dbudworth> Trying to figure out if 0mq is appropriate for my project, simple description would be something like a sticky load balancer. once a client talks to a service, it sticks with that one. for a bi-directional conversation between 2 nodes once an initial node has been selected. ie: my client does round robin on each new conversation but updates to a given conversation have to stick to the beginning server
[Sunday 19 September 2010] [23:27:43] <dbudworth> and hello stoneman, i’m here. but basically not useful if you are looking for help
[Sunday 19 September 2010] [23:35:48] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [23:36:07] Part stoneman has left this channel.
[Monday 20 September 2010] [01:08:51] Quit guido_g has left this server (Quit: Boom...).
[Monday 20 September 2010] [01:33:38] Quit jsimmons has left this server (Ping timeout: 264 seconds).
[Monday 20 September 2010] [01:56:23] Quit keffo has left this server (Ping timeout: 245 seconds).
[Monday 20 September 2010] [02:22:21] Quit benford has left this server (Quit: benford).
[Monday 20 September 2010] [02:54:04] Part benford has left this channel.
[Monday 20 September 2010] [03:04:31] Quit kleppari has left this server (Ping timeout: 246 seconds).
[Monday 20 September 2010] [03:06:28] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Monday 20 September 2010] [04:22:49] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 20 September 2010] [05:10:23] <mato> sustrik: i was just thinking about the zmq_term()/zmq_close() semantics
[Monday 20 September 2010] [05:10:40] <mato> sustrik: and i came up with one small detail
[Monday 20 September 2010] [05:11:03] <mato> sustrik: which may or may not be helpful in the case where zmq_term() would wait indefinitely on exit
[Monday 20 September 2010] [05:11:23] <mato> sustrik: the thing is, syscall close() means “commit”.
[Monday 20 September 2010] [05:12:02] <mato> sustrik: if you don’t close your filedescriptors, and don’t use fsync() (not for sockets obviously), then there is no commitment on the part of the kernel
[Monday 20 September 2010] [05:13:02] <mato> sustrik: hence, as far as sockets with data in-flight at zmq_term() time are concerned, we only need care about those that zmq_close() has actually been called on
[Monday 20 September 2010] [05:13:10] <mato> sustrik: for any other sockets, all bets are off
[Monday 20 September 2010] [05:19:00] <sustrik> mato: sure, but user has to close all the sockets anyway
[Monday 20 September 2010] [05:19:12] <sustrik> otherwise he’ll end up with memory leaks
[Monday 20 September 2010] [05:19:52] <mato> sustrik: i was thinking more of the case of “the application terminating in an abnormal way”
[Monday 20 September 2010] [05:20:03] <sustrik> no guarantees then
[Monday 20 September 2010] [05:20:14] <mato> sure, what i mean is...
[Monday 20 September 2010] [05:20:24] <mato> if an application handles e.g. SIGINT/SIGTERM
[Monday 20 September 2010] [05:20:39] <mato> should it go and call zmq_term() as part of its exit process in such a case?
[Monday 20 September 2010] [05:20:48] <mato> or explicitly not do that, becuase the call may block...?
[Monday 20 September 2010] [05:21:02] <mato> i guess the answer is application dependent
[Monday 20 September 2010] [05:21:21] <sustrik> no idea
[Monday 20 September 2010] [05:21:48] <mato> oh, and another thing
[Monday 20 September 2010] [05:21:58] <mato> we talked about the non-zero-copy APIs
[Monday 20 September 2010] [05:22:10] <mato> and i have the most obvious names :)
[Monday 20 September 2010] [05:22:22] <mato> zmq_sendcopy and zmq_recvcopy ... ?
[Monday 20 September 2010] [05:22:43] <mato> would do fine for 2.1, in 3.x the whole mess can be done the right way
[Monday 20 September 2010] [05:23:31] <sustrik> dunno, does it makes sense to introduce something that will be changed anyway?
[Monday 20 September 2010] [05:23:45] <sustrik> it’s just asking for problems
[Monday 20 September 2010] [05:23:50] <mato> why?
[Monday 20 September 2010] [05:24:41] <mato> well, one reason is that pieter uses all these “helper” functions in the user guide precisely for this reason
[Monday 20 September 2010] [05:25:07] <sustrik> so, we’ll have send, sendcopy and the helper function
[Monday 20 September 2010] [05:25:16] <sustrik> :)
[Monday 20 September 2010] [05:25:16] <mato> no, the helpers can go away
[Monday 20 September 2010] [05:25:36] <mato> check with pieter, but imo the reason those are there is becuase zerocopy in C is “too much typing” or something :)
[Monday 20 September 2010] [05:25:43] <mato> at least that’s what he told me
[Monday 20 September 2010] [05:25:54] <sustrik> anyway, first thing on this path is to defined the semantics for recvcopy when the buffer is no large enough
[Monday 20 September 2010] [05:26:06] * sustrik has to have a look at sctp
[Monday 20 September 2010] [05:27:10] <mato> yeah, good point
[Monday 20 September 2010] [05:27:28] <mato> presumably some error will have to be returned ...
[Monday 20 September 2010] [05:27:47] <mato> SCTP works in terms of messages as atomic units?
[Monday 20 September 2010] [05:29:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [05:30:28] <sustrik> mato: yes
[Monday 20 September 2010] [08:05:11] Quit mitkok has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [08:14:24] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [08:16:38] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 20 September 2010] [09:03:10] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [09:03:58] Quit dbudworth has left this server (Quit: dbudworth).
[Monday 20 September 2010] [09:25:09] Quit guido_g has left this server (Quit: Boom...).
[Monday 20 September 2010] [09:36:51] <user> was hoping someone can help me
[Monday 20 September 2010] [09:37:43] <user> cant download anything always comes up with an error
[Monday 20 September 2010] [09:44:18] <user> whats that?
[Monday 20 September 2010] [09:44:43] <drbobbeaty> The POSIX version is the version for Linux, etc. There’s also a Windows version on the same page.
[Monday 20 September 2010] [09:44:47] <mato> sustrik: are you around?
[Monday 20 September 2010] [09:45:20] <mato> sustrik: I’ve found a silly bug in the select() impl. of zmq_poll() on master
[Monday 20 September 2010] [09:45:31] <sustrik> mato: here am I
[Monday 20 September 2010] [09:45:35] <mato> sustrik: trivial fix, but something else is broken when using select()
[Monday 20 September 2010] [09:45:44] <user> k. my error code tells me: Archive: /media/OFFICE12/setup.exe
[Monday 20 September 2010] [09:45:44] <user> [/media/OFFICE12/setup.exe]
[Monday 20 September 2010] [09:45:44] <user> End-of-central-directory signature not found. Either this file is not
[Monday 20 September 2010] [09:45:44] <user> a zipfile, or it constitutes one disk of a multi-part archive. In the
[Monday 20 September 2010] [09:45:44] <user> latter case the central directory and zipfile comment will be found on
[Monday 20 September 2010] [09:45:45] <user> the last disk(s) of this archive.
[Monday 20 September 2010] [09:45:47] <user> note: /media/OFFICE12/setup.exe may be a plain executable, not an archive
[Monday 20 September 2010] [09:45:49] <user> zipinfo: cannot find zipfile directory in one of /media/OFFICE12/setup.exe or
[Monday 20 September 2010] [09:45:51] <user> /media/OFFICE12/setup.exe.zip, and cannot find /media/OFFICE12/setup.exe.ZIP, period.
[Monday 20 September 2010] [09:46:25] <mato> user: I’m sorry, you’re probably on the wrong chat room here.
[Monday 20 September 2010] [09:46:38] <user> where do i need to go?
[Monday 20 September 2010] [09:47:15] <sustrik> :)
[Monday 20 September 2010] [09:47:46] <sustrik> mato: wull?
[Monday 20 September 2010] [09:48:02] <mato> sustrik: wull?
[Monday 20 September 2010] [09:48:11] <sustrik> well?
[Monday 20 September 2010] [09:48:17] <mato> hang on
[Monday 20 September 2010] [09:48:20] <mato> i’m on bloody win32
[Monday 20 September 2010] [09:48:25] <mato> everything is confusing :-)
[Monday 20 September 2010] [09:48:28] <mato> patience...
[Monday 20 September 2010] [09:49:28] <mato> sustrik: ok, 1st, somewhee around line 547 of zmq.cpp, the select () call needs to be changed to use maxfd + 1
[Monday 20 September 2010] [09:49:32] <mato> sustrik: not just maxfd
[Monday 20 September 2010] [09:49:37] <mato> sustrik: that’s probably my fault
[Monday 20 September 2010] [09:49:43] <sustrik> ok
[Monday 20 September 2010] [09:50:26] <mato> sustrik: now, then, what i’m seeing is on win32 _or_ on Linux with ZMQ_FORCE_SELECT (and I patched zmq.cpp to also use select() for zmq_poll() when zmq_force_select() is defined)
[Monday 20 September 2010] [09:50:52] <mato> sustrik: for some reason a socket is not becoming ready on the app side when it should
[Monday 20 September 2010] [09:51:34] <mato> sustrik: i.e. data gets sent down XREP, XREQ on the client side never becomes ready
[Monday 20 September 2010] [09:51:58] <sustrik> do you have a simple test program?
[Monday 20 September 2010] [09:52:06] <mato> working on it...
[Monday 20 September 2010] [09:52:52] <sustrik> ok
[Monday 20 September 2010] [09:54:35] Part user has left this channel (“Leaving”).
[Monday 20 September 2010] [10:20:07] <mato> sustrik: ok, i have a test case... will msg you, it;s on the test box
[Monday 20 September 2010] [10:45:13] <CIA-20> zeromq2: 03Martin Lucina 07master * r1abfc92 10/ src/zmq.cpp : minor problem in zmq_poll (select version) fixed -
http://bit.ly/c9Sdnb[Monday 20 September 2010] [10:55:34] <CIA-20> zeromq2: 03Martin Lucina 07master * rf49b77e 10/ src/zmq.cpp : zmq_poll honours ZMQ_FORCE_POLL and ZMQ_FORCE_SELECT options -
http://bit.ly/dzi76e[Monday 20 September 2010] [10:56:23] Quit GeekGod has left this server (Quit: GeekGod).
[Monday 20 September 2010] [11:16:12] <sustrik> mato: afaics the problem is that ZMQ_FD is edge-trigerred
[Monday 20 September 2010] [11:17:26] <sustrik> thus, IN/OUT flag may be set in the past, but the select/poll won’t exit because of it
[Monday 20 September 2010] [11:18:32] <sustrik> mato: check how poll version of zmq_poll works
[Monday 20 September 2010] [11:20:32] <mato> sustrik: the code seems equivalent, no?
[Monday 20 September 2010] [11:20:36] <sustrik> nope
[Monday 20 September 2010] [11:21:02] <mato> then i don’t understand the problem...
[Monday 20 September 2010] [11:21:05] <sustrik> you have to delete lines 572.577
[Monday 20 September 2010] [11:21:11] <sustrik> let me do it
[Monday 20 September 2010] [11:21:58] <mato> which lines, i have different line numbers here...
[Monday 20 September 2010] [11:22:12] <mato> and i’d like to understand what the problem is
[Monday 20 September 2010] [11:22:46] <sustrik> you should _not_ check POLLIN is set on ZMQ_FD and check ZMQ_EVENTS anyway
[Monday 20 September 2010] [11:22:56] <sustrik> whather
[Monday 20 September 2010] [11:22:59] <sustrik> ehether
[Monday 20 September 2010] [11:23:01] <sustrik> whether
[Monday 20 September 2010] [11:23:02] <sustrik> :)
[Monday 20 September 2010] [11:23:03] <mato> ?
[Monday 20 September 2010] [11:23:16] <sustrik> ...whether POLLIN is set...
[Monday 20 September 2010] [11:23:48] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [11:24:44] <mato> sustrik: hmm, so you’re saying that each ZMQ_FD needs to be checked every time you come out of the select/poll() ?
[Monday 20 September 2010] [11:24:48] <CIA-20> zeromq2: 03Martin Sustrik 07master * r4d51a52 10/ src/zmq.cpp : zmq_poll (select version) now correctly assumes that ZMQ_FD is edge-trigerred -
http://bit.ly/dklauP[Monday 20 September 2010] [11:25:07] <sustrik> yes
[Monday 20 September 2010] [11:25:15] <sustrik> committed
[Monday 20 September 2010] [11:25:23] <sustrik> the test program seems to work
[Monday 20 September 2010] [11:26:23] <mato> i still don’t understand... select would not have exited if the fd did not become ready...?
[Monday 20 September 2010] [11:26:41] Quit jsimmons has left this server (Ping timeout: 276 seconds).
[Monday 20 September 2010] [11:27:26] <sustrik> what?
[Monday 20 September 2010] [11:27:51] <mato> sustrik: if you’re sitting in select(), and the notify fd becomes ready, then you read the events
[Monday 20 September 2010] [11:27:59] <mato> sustrik: what is the other code path?
[Monday 20 September 2010] [11:28:18] <sustrik> the commands in ZMQ_FD were already processed before calling zmq_poll
[Monday 20 September 2010] [11:28:25] <sustrik> select blocks forever
[Monday 20 September 2010] [11:28:38] <sustrik> although there are messages in available
[Monday 20 September 2010] [11:28:40] <mato> processed by who?
[Monday 20 September 2010] [11:28:51] <sustrik> random previous command
[Monday 20 September 2010] [11:29:12] <mato> ah, right, this is because ZMQ_FD is tapping straight into the signaller
[Monday 20 September 2010] [11:29:15] <mato> mumble
[Monday 20 September 2010] [11:29:20] <sustrik> ack
[Monday 20 September 2010] [11:29:22] <mato> i wish we could fix that
[Monday 20 September 2010] [11:29:27] <sustrik> ?
[Monday 20 September 2010] [11:29:33] <sustrik> it’s fixed
[Monday 20 September 2010] [11:29:42] <mato> this is also why you do that first_pass thing, right?
[Monday 20 September 2010] [11:29:47] <sustrik> yes
[Monday 20 September 2010] [11:29:55] <sustrik> no timout on first pass
[Monday 20 September 2010] [11:30:04] <sustrik> exit immediately
[Monday 20 September 2010] [11:30:15] <sustrik> then check whether events are availalbe
[Monday 20 September 2010] [11:30:25] <mato> this is to pick up events coming from previously processed commands, right?
[Monday 20 September 2010] [11:30:34] <sustrik> yes
[Monday 20 September 2010] [11:30:40] <mato> ok, understood
[Monday 20 September 2010] [11:31:00] * mato will re-test and see what happens
[Monday 20 September 2010] [11:38:39] <sustrik> mato: btw, if you want to optimise it you can still perform the check when !first_pass
[Monday 20 September 2010] [11:39:16] <sustrik> getting ZMQ_EVENTS can be rather slow as it involves reading from the signaler => recv()
[Monday 20 September 2010] [11:40:40] <mato> right, that might be a good idea
[Monday 20 September 2010] [11:40:48] <mato> also, is getting ZMQ_FD slow?
[Monday 20 September 2010] [11:41:01] <mato> since that is also done more times than strictly necessary
[Monday 20 September 2010] [11:41:13] <mato> sustrik: btw, the signaler uses socketpair?
[Monday 20 September 2010] [11:41:32] <mato> sustrik: which translates to a tcp connection with winsock?
[Monday 20 September 2010] [11:44:30] <sustrik> yes
[Monday 20 September 2010] [11:44:54] <mato> interesting, i get an occasional “Address already in use” on Win32 from signaler.cpp:80
[Monday 20 September 2010] [11:45:06] <sustrik> let me see
[Monday 20 September 2010] [11:45:18] <mato> I guess this is just the poor M$ WIn$ock running out of tcp ports or something
[Monday 20 September 2010] [11:45:27] <mato> it’s the “connect to remote peer” call...
[Monday 20 September 2010] [11:45:34] <sustrik> EADDRINUSE on connect?
[Monday 20 September 2010] [11:45:52] <mato> yeah
[Monday 20 September 2010] [11:45:56] <mato> bizarre
[Monday 20 September 2010] [11:46:04] <sustrik> hm, it’s documented
[Monday 20 September 2010] [11:46:16] <mato> “not enough ports” ?
[Monday 20 September 2010] [11:47:01] <sustrik> very useful description:
[Monday 20 September 2010] [11:47:02] <sustrik> [EADDRINUSE]
[Monday 20 September 2010] [11:47:02] <sustrik> Attempt to establish a connection that uses addresses that are already in use.
[Monday 20 September 2010] [11:47:30] Quit xla has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [11:47:33] <sustrik> (that’s POSIX)
[Monday 20 September 2010] [11:48:00] <mato> sustrik: windoze is slightly different
[Monday 20 September 2010] [11:48:05] <sustrik> Linux: EADDRINUSE
[Monday 20 September 2010] [11:48:05] <sustrik> Local address is already in use.
[Monday 20 September 2010] [11:49:17] <sustrik> The socket’s local address is already in use and the socket was not marked to allow address reuse with SO_REUSEADDR. This error usually occurs when executing bind, but could be delayed until the connect function if the bind was to a wildcard address (INADDR_ANY or in6addr_any) for the local IP address. A specific address needs to be implicitly bound by the connect function.
[Monday 20 September 2010] [11:49:40] <mato> Yeah, I never quite understood the SO_REUSEADDR semantics on win32
[Monday 20 September 2010] [11:49:47] <sustrik> does it make any sense to you?
[Monday 20 September 2010] [11:49:54] <mato> but i’ll try adding that in and see if it changes anything
[Monday 20 September 2010] [11:50:19] <mato> well, i think what it’s saying is “the wildcard bind() picked a port that is still in time-wait state”
[Monday 20 September 2010] [11:50:27] <mato> or some nonsense like that
[Monday 20 September 2010] [11:50:31] <sustrik> :|
[Monday 20 September 2010] [11:50:42] <mato> let me try adding SO_REUSEADDR for win32 to the signaler listen socket and see what happens
[Monday 20 September 2010] [11:50:57] <mato> i don’t care what it actually does on win32 as long as the error goes away :-)
[Monday 20 September 2010] [11:51:25] <mato> proper win32 solution is obviously to use named pipes/win32 objects/whatsits and iocp
[Monday 20 September 2010] [11:54:09] <sustrik> ok
[Monday 20 September 2010] [12:07:35] <mato> hmm
[Monday 20 September 2010] [12:07:38] <mato> i dunno
[Monday 20 September 2010] [12:07:42] <mato> doesn’t seem to help much
[Monday 20 September 2010] [12:07:47] <mato> anyway, this doesn’t really matter
[Monday 20 September 2010] [12:08:11] <mato> i added SO_REUSEADDR to both ends of the emulated socketpair and still get EADDRINUSE back
[Monday 20 September 2010] [12:08:27] <mato> so i think it’s just poor windows running out of ports to auto-assign or something :-)
[Monday 20 September 2010] [12:08:50] <mato> sustrik: anyway, the problem with select() has been fixed, so it’s all good!
[Monday 20 September 2010] [12:11:05] <sustrik> nice
[Monday 20 September 2010] [12:11:26] <sustrik> you may fill in the bug report for the EADDRINUSE stuff
[Monday 20 September 2010] [12:24:30] <mato> done
[Monday 20 September 2010] [12:45:23] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Monday 20 September 2010] [13:07:13] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Monday 20 September 2010] [13:14:11] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Monday 20 September 2010] [13:18:32] Quit psino has left this server (Quit: psino).
[Monday 20 September 2010] [13:38:12] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Monday 20 September 2010] [13:50:52] Quit mivert has left this server (Quit: Leaving).
[Monday 20 September 2010] [14:28:40] Quit omarkj has left this server (Quit: omarkj).
[Monday 20 September 2010] [14:52:12] Quit mitkok has left this server (Read error: No route to host).
[Monday 20 September 2010] [14:55:48] Quit mitkok1 has left this server (Client Quit).
[Monday 20 September 2010] [15:13:59] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 20 September 2010] [15:54:16] Quit kelvink has left this server (Quit: Leaving.).
[Monday 20 September 2010] [16:00:52] <cpscotti> Hello there, anyone up to a philosophical (although basic) discussion on a zmq networking topology for a “generic” application? Regarding many clients & services but without a broker.
[Monday 20 September 2010] [16:03:43] Quit jhawk28 has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [16:07:18] Quit mitkok has left this server (Client Quit).
[Monday 20 September 2010] [16:08:37] Nick mitkok is now known as mytrile.
[Monday 20 September 2010] [16:17:01] <cremes> cpscotti: i’d be happy to have that conversation with you tomorrow; i have to leave for the rest of today
[Monday 20 September 2010] [16:19:20] <cpscotti> cremes: thanks.. if nothing is solved in my side of things I’ll try tomorrow then
[Monday 20 September 2010] [16:19:50] <cremes> ok
[Monday 20 September 2010] [16:32:22] Quit mytrile has left this server (Read error: No route to host).
[Monday 20 September 2010] [16:34:27] Quit mitkok has left this server (Client Quit).
[Monday 20 September 2010] [17:07:39] <ModusPwnens> is the send function faster than the receive function? or are they both just as fast?
[Monday 20 September 2010] [17:08:10] <cpscotti> send doesn’t block, recv blocks
[Monday 20 September 2010] [17:08:21] <cpscotti> (as for the “program flow” speed)
[Monday 20 September 2010] [17:08:33] <cpscotti> now for the underlying stuff, dunno
[Monday 20 September 2010] [17:08:36] <ModusPwnens> even with pub/sub topology?
[Monday 20 September 2010] [17:08:37] <jhawk28> recv can do a noblock flag
[Monday 20 September 2010] [17:08:49] <cpscotti> awl.. yep..
[Monday 20 September 2010] [17:08:56] <ModusPwnens> I see.
[Monday 20 September 2010] [17:09:09] <ModusPwnens> So that would explain benchmarking tests where
[Monday 20 September 2010] [17:09:12] <ModusPwnens> i run several in a row
[Monday 20 September 2010] [17:09:59] <ModusPwnens> thats probably better than explaining it
[Monday 20 September 2010] [17:10:37] <ModusPwnens> I’ve sort of run into a wall with this and have been stuck on it for several days
[Monday 20 September 2010] [17:10:44] <ModusPwnens> because I am seeing those results
[Monday 20 September 2010] [17:10:47] <ModusPwnens> and am not sure what to make of them
[Monday 20 September 2010] [18:10:12] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [18:26:19] Quit cpscotti has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [18:30:32] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 20 September 2010] [18:35:19] Quit mytrile has left this server (Quit: Leaving.).
[Monday 20 September 2010] [18:41:15] <larrytheliquid> are there any semantic implications to connecting multiple times to the same address?
[Monday 20 September 2010] [18:41:36] <larrytheliquid> from the same socket, that is
[Monday 20 September 2010] [18:44:18] Quit sbahra has left this server (Remote host closed the connection).
[Monday 20 September 2010] [18:45:20] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 20 September 2010] [18:49:06] Quit gavinstark has left this server (Remote host closed the connection).
[Monday 20 September 2010] [18:53:43] Quit dbudworth has left this server (Quit: dbudworth).
[Monday 20 September 2010] [19:02:09] Quit ModusPwnens has left this server.
[Monday 20 September 2010] [19:31:31] Quit Samy has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [19:36:02] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Monday 20 September 2010] [19:38:33] Join Samy has joined this channel (
~sbahra@8.8.38.2).
[Monday 20 September 2010] [20:58:34] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Monday 20 September 2010] [21:17:48] Quit andrewvc has left this server (Quit: andrewvc).
*** on Mon Sep 20 21:52:23 2010
[Monday 20 September 2010] [21:52:23] Topic The channel topic is “Welcome!”.
[Monday 20 September 2010] [21:52:27] Mode Channel modes: no messages from outside, topic protection
[Monday 20 September 2010] [21:52:27] Created This channel was created on 2010-02-05 10:44.
[Monday 20 September 2010] [22:15:40] Quit jhawk28 has left this server (Remote host closed the connection).
[Monday 20 September 2010] [23:10:53] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 21 September 2010] [00:22:53] Quit iFire has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [01:29:13] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [01:37:27] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 21 September 2010] [01:43:53] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [01:45:30] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Tuesday 21 September 2010] [01:55:18] Quit grahamalot has left this server (Quit: grahamalot).
[Tuesday 21 September 2010] [03:03:31] Quit benford has left this server (Quit: benford).
[Tuesday 21 September 2010] [03:04:26] Quit kleppari has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [03:10:35] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Tuesday 21 September 2010] [03:36:36] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [04:05:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * r9c11886 10/ (src/socket_base.cpp src/socket_base.hpp): The flag in the socket has clear semantics now – it tracks whether corresponding context was closed, it doesn’t track whether zmq_close was called on the socket itself -
http://bit.ly/d6ktZI[Tuesday 21 September 2010] [04:05:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * re478468 10/ src/socket_base.hpp : minor error in comment fixed -
http://bit.ly/a4RuQJ[Tuesday 21 September 2010] [04:45:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [04:50:56] Quit xla has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [04:52:20] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [05:20:42] Quit xla has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [05:22:11] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [05:25:11] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [06:37:38] Quit SecretAgent has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [07:19:50] Quit timothyfitz has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [08:52:37] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [09:38:31] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [10:30:52] Quit mytrile has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [10:33:26] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 21 September 2010] [11:23:17] Part sd88g93 has left this channel.
[Tuesday 21 September 2010] [11:56:30] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [11:56:52] <starkdg> hello
[Tuesday 21 September 2010] [11:57:09] <guido_g> hi
[Tuesday 21 September 2010] [12:12:06] <CIA-20> rbzmq: 03Brian Buchanan 07master * rdc552c2 10/ rbzmq.c : Make PUSH and PULL constants work even if using an older version of the library. -
http://bit.ly/9ZisAu[Tuesday 21 September 2010] [12:12:08] <CIA-20> rbzmq: 03Brian Buchanan 07master * rf17f6ca 10/ zmq.gemspec : update gem version to 2.0.9 -
http://bit.ly/b4wU0A[Tuesday 21 September 2010] [12:21:58] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [13:02:20] Quit RobertLJ has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [13:24:48] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [13:53:42] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 21 September 2010] [13:56:46] Quit starkdg has left this server (Quit: goodbye).
[Tuesday 21 September 2010] [14:05:19] Quit xla has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [14:19:27] <AltraMayor> Hi there,
[Tuesday 21 September 2010] [14:20:06] <AltraMayor> I’ve been reading about PNUTS, a NoSQL solution written by Yahoo!
[Tuesday 21 September 2010] [14:21:21] <AltraMayor> A very important element of their design is their message brokers, which guarantee that messages aren’t lost. For example, they can make brokers replicate messages with others to support failure.
[Tuesday 21 September 2010] [14:21:45] <AltraMayor> Is this use case supported by zeromq?
[Tuesday 21 September 2010] [14:21:55] <guido_g> no, zeromq doesn’t have that
[Tuesday 21 September 2010] [14:22:34] <AltraMayor> Thanks, guido_g.
[Tuesday 21 September 2010] [14:23:21] <AltraMayor> I’ve read that page, but since it says that’s more coming, I couldn’t assume that it’s all.
[Tuesday 21 September 2010] [14:23:54] <AltraMayor> Does anyone know a message broker designed to support that level of failure?
[Tuesday 21 September 2010] [14:24:09] <guido_g> you m ight have skipped the important things
[Tuesday 21 September 2010] [14:24:31] <AltraMayor> I don’t follow you, guido_g.
[Tuesday 21 September 2010] [14:24:33] <guido_g> like that mq doesn’t have or is a message broker
[Tuesday 21 September 2010] [14:25:11] <AltraMayor> I understand that zeromq isn’t a broker, but its durable option isn’t enough.
[Tuesday 21 September 2010] [14:25:41] <cremes> AltraMayor: the roadmap is a wiki page that can be edited by anyone who signs up for an account
[Tuesday 21 September 2010] [14:25:52] <cremes> if you think there are things missing, feel free to add them
[Tuesday 21 September 2010] [14:25:55] <AltraMayor> I also understood I could implement a broker as reliable as I want using zeromq, but this would be just a piece of what I’m looking for.
[Tuesday 21 September 2010] [14:29:12] <AltraMayor> Cremes: what’s the link to the roadmap page you mentioned?
[Tuesday 21 September 2010] [14:31:55] <AltraMayor> Cremes: do you think there’s interest among the developers to implement this level of robustness? As I read, zeroqm is implemented by iMatix to meet their demands, I don’t know if they’d be interested in a wish list of a guy that wants to have an open-source alternative to PNUTS.
[Tuesday 21 September 2010] [14:32:25] <cremes> good point... i heard their ceo speak on that very topic yesterday
[Tuesday 21 September 2010] [14:32:34] <cremes> if you write the code, it will get done
[Tuesday 21 September 2010] [14:32:41] <cremes> if you pay them to write the code, it will get done
[Tuesday 21 September 2010] [14:35:03] <AltraMayor> I understand, but I’m looking for pieces to estimate the total effort to have an opensource PNUTS.
[Tuesday 21 September 2010] [14:36:37] <cremes> okay, well i don’t know what pnuts is :)
[Tuesday 21 September 2010] [14:39:57] <AltraMayor> Cremes: can you provide me a small description how you use 0mq?
[Tuesday 21 September 2010] [14:41:04] <cremes> AltraMayor: sure...
[Tuesday 21 September 2010] [14:41:24] <cremes> i use it to tie together a bunch of distributed components; i used to use amqp but threw it away for 0mq
[Tuesday 21 September 2010] [14:41:40] <cremes> i do some pub/sub and some request/reply style message passing
[Tuesday 21 September 2010] [14:41:55] <cremes> that’s the short version
[Tuesday 21 September 2010] [14:42:01] <AltraMayor> How do your components deal with failure? Do you just restart them?
[Tuesday 21 September 2010] [14:42:27] <cremes> it depends
[Tuesday 21 September 2010] [14:42:45] <cremes> if certain components fail, it is fatal for the whole system
[Tuesday 21 September 2010] [14:42:58] <AltraMayor> By the way, PNUTS is a NoSQL that supports replication/accesses/updates among datacenters.
[Tuesday 21 September 2010] [14:43:05] <cremes> if others fail, i detect their absence and just restart/resend their jobs to surviving nodes
[Tuesday 21 September 2010] [14:43:32] <cremes> then perhaps you want to look at mongodb as something similar; 0mq has nothing to do with nosql
[Tuesday 21 September 2010] [14:43:40] <cremes> it’s not a database in any way, shape or form
[Tuesday 21 September 2010] [14:47:47] <AltraMayor> I see that 0mq isn’t a database, but messaging brokers are key in PNUTS’ design. Mongodb is the best NoSQL that I know so far, but it don’t support datacenters as graceful as PNUTS. And, I’m considering implement some pieces, I just didn’t want to start from so low as the message broker.
[Tuesday 21 September 2010] [14:48:36] <guido_g> then get something w/ a broker
[Tuesday 21 September 2010] [14:49:36] <AltraMayor> Ok.
[Tuesday 21 September 2010] [14:50:08] <AltraMayor> Thank you for your time!
[Tuesday 21 September 2010] [14:50:39] Quit AltraMayor has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [14:50:52] <guido_g> best would be broker which stores the messages in a reliable way distributed among datacenters
[Tuesday 21 September 2010] [14:51:28] <guido_g> now wating for the stack overflow...
[Tuesday 21 September 2010] [15:30:36] Quit cpscotti has left this server (Ping timeout: 252 seconds).
[Tuesday 21 September 2010] [15:49:18] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [16:25:49] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [16:56:11] <Samy> like the new website
[Tuesday 21 September 2010] [16:56:49] <Samy> It would be nice if there was a more obvious link to the source repository.
[Tuesday 21 September 2010] [16:59:06] <Samy> Ah, zeromq.com != zeromq.org
[Tuesday 21 September 2010] [17:07:55] <mato> Samy: yeah, the .com site is intended as a portal for new users
[Tuesday 21 September 2010] [17:08:09] <mato> Samy: those who just want a first quick look
[Tuesday 21 September 2010] [17:11:21] <jond> mato: hi
[Tuesday 21 September 2010] [17:14:31] <mato> jond: hi there
[Tuesday 21 September 2010] [17:15:04] <jond> mato: i’ve been looking at the select.cpp and i think there is an issue
[Tuesday 21 September 2010] [17:15:23] <jond> because I’m fixing something similar in the dayjob
[Tuesday 21 September 2010] [17:17:08] <mato> jond: what kind of issue?
[Tuesday 21 September 2010] [17:17:14] <jond> as well as checking that the array doesnt get to FD_SETSIZE, it needs to check that the
fd_ in add_fd is actually greater than FD_SETSIZE
[Tuesday 21 September 2010] [17:18:45] <mato> why would the actual fd matter?
[Tuesday 21 September 2010] [17:18:49] <jond> FD_SET, CLR etc don’t work on linux above FD_SETSIZE. it’s on the man page at the bottom; I had a process with 6000 fd’s open and once the sockets started connecting and getting fd > 1024 bad things happen
[Tuesday 21 September 2010] [17:19:31] <jond> fd_set is normally implemented as a bitvector of fixed size
[Tuesday 21 September 2010] [17:19:33] <mato> ah, you’re right
[Tuesday 21 September 2010] [17:19:35] <mato> yeah
[Tuesday 21 September 2010] [17:19:46] <mato> good point
[Tuesday 21 September 2010] [17:20:21] <mato> well, the select implementation has not really been stress-tested properly
[Tuesday 21 September 2010] [17:20:40] <mato> it’s only there for windows benefit (and vms I guess)
[Tuesday 21 September 2010] [17:20:52] <jond> we raised call with redhat and even if you recompile glibc etc the kernel has it at 1024. My problem is that the 6000 fd’s are files opened early and the sockets connect later and get to high fd.
[Tuesday 21 September 2010] [17:21:02] <jond> i’m changing to poll
[Tuesday 21 September 2010] [17:21:27] <mato> well, if you’re dealing with 000’s of fds you really should not be using select at all :-)
[Tuesday 21 September 2010] [17:21:28] <jond> on solaris I believe you can just redefine FD_SETSIZE
[Tuesday 21 September 2010] [17:21:48] <mato> and poll is not too performant with lots of fds either, you really want an edge-triggered interface
[Tuesday 21 September 2010] [17:22:27] <mato> anyhow, thanks for the tip; i’ll make a note to make a patch that adds in the appropriate assertions
[Tuesday 21 September 2010] [17:23:03] <jond> yep I know, but I ‘inherited’ this code; we only have a 600 tcp connections, but open lots of files....
[Tuesday 21 September 2010] [17:24:38] <jond> mato: edge-triggered, you mean epoll?
[Tuesday 21 September 2010] [17:27:02] <mato> jond: yes
[Tuesday 21 September 2010] [17:27:21] <mato> jond: unfortunately epoll never got standardised so is available only on Linux
[Tuesday 21 September 2010] [17:27:42] <mato> jond:
*BSD has kqueue, Solaris has Event Completion
[Tuesday 21 September 2010] [17:27:53] <jond> i’ll take a look tommorrow; thanks for the tip;
[Tuesday 21 September 2010] [17:28:09] <jond> mato: cheers
[Tuesday 21 September 2010] [17:36:42] <bgranger> mato: did you see my reply about the message related deadlock you were seeing?
[Tuesday 21 September 2010] [17:38:31] <jond> bgranger: i did. is there a link to that pyzmq branch
[Tuesday 21 September 2010] [17:38:39] <bgranger> Sure, right here...
[Tuesday 21 September 2010] [17:38:54] <dermoth> Is there any example of XREQ/XREP out there? I’ve used REP/REP successfully in the past and now I’d like to add devices in the middle, and I believe I need XREQ/XREP for that.
[Tuesday 21 September 2010] [17:39:11] <bgranger> Here is the class
[Tuesday 21 September 2010] [17:40:03] <bgranger> We are still doing code review so a few more things might change in the API, but I think things are mostly in place.
[Tuesday 21 September 2010] [17:40:49] <mato> bgranger: hi
[Tuesday 21 September 2010] [17:41:04] <bgranger> hi
[Tuesday 21 September 2010] [17:41:12] <mato> bgranger: yes, i saw that, am travelling most of this week (Linux Kongress in Nurnberg, Martin Sustrik is presenting)
[Tuesday 21 September 2010] [17:41:21] <bgranger> Nice
[Tuesday 21 September 2010] [17:41:30] <jond> bgranger: thanks, i’ll take a look. I think that function that’s added in py2.7 for PendingCall looks interesting and could still provide a version which doesnt need the MesssageTrcaker
[Tuesday 21 September 2010] [17:41:33] <mato> bgranger: will look at it next week once I get back home
[Tuesday 21 September 2010] [17:41:48] <bgranger> Great, sounds good, by then it will probably be in our master.
[Tuesday 21 September 2010] [17:41:50] <mato> bgranger: can’t really test it reliably while on the road.
[Tuesday 21 September 2010] [17:41:59] <bgranger> OK
[Tuesday 21 September 2010] [17:53:17] Quit mytrile has left this server (Read error: Connection reset by peer).
*** on Tue Sep 21 17:58:29 2010
[Tuesday 21 September 2010] [17:58:29] Topic The channel topic is “Welcome!”.
[Tuesday 21 September 2010] [17:58:33] Mode Channel modes: no messages from outside, topic protection
[Tuesday 21 September 2010] [17:58:33] Created This channel was created on 2010-02-05 10:44.
[Tuesday 21 September 2010] [17:58:43] <dermoth> [bbl]
[Tuesday 21 September 2010] [18:44:02] Quit jhawk28 has left this server (Ping timeout: 265 seconds).
[Tuesday 21 September 2010] [18:51:17] Quit mytrile has left this server (Ping timeout: 265 seconds).
[Tuesday 21 September 2010] [19:27:45] <tylergillies> whats the difference between req/rep and upstream/downstream?
[Tuesday 21 September 2010] [19:31:11] <vanadium> req/rep are bidirectional/back and forth, upstream/downstream are push/pull and one of them always sends and the other always receives
[Tuesday 21 September 2010] [19:38:36] Quit dbudworth has left this server (Quit: dbudworth).
[Tuesday 21 September 2010] [19:46:19] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 21 September 2010] [20:27:15] Part starkdg has left this channel.
[Tuesday 21 September 2010] [20:32:04] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [20:43:19] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [20:59:19] Quit
jhawk28_ has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [21:03:26] <dermoth> actually one of the issues that I was seeing is obvious... in a php worker I want to have a persistent socket that use req/rep (or the x equivalent if needed), but if the backend isn’T responding timely I just want to leave early and go on, but then when the next request comes in the socket is still expecting a reply... how can I work around this?
[Tuesday 21 September 2010] [21:03:41] <dermoth> i guess that’S a good one for the ML :)
[Tuesday 21 September 2010] [21:05:59] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 21 September 2010] [21:18:31] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Tuesday 21 September 2010] [22:10:36] Quit jhawk28 has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [00:01:14] Quit RobertLJ has left this server (Ping timeout: 264 seconds).
[Wednesday 22 September 2010] [00:19:54] Quit kleppari has left this server (Ping timeout: 272 seconds).
*** on Wed Sep 22 07:12:46 2010
[Wednesday 22 September 2010] [07:12:46] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [07:12:50] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [07:12:50] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [07:53:53] <raz> hm
[Wednesday 22 September 2010] [07:53:58] <raz> looking at the divide&conquer example i wonder
[Wednesday 22 September 2010] [07:54:18] <raz> how does zmq behave when a worker does not indicate it finished? (e.g. because it crashed)
[Wednesday 22 September 2010] [07:54:25] <raz> will the task be re-queued to a different worker?
[Wednesday 22 September 2010] [08:00:10] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Wednesday 22 September 2010] [08:00:10] Notice -NickServ- You are now identified for travlr.
[Wednesday 22 September 2010] [08:00:10] CTCP Received Version request from frigg.
[Wednesday 22 September 2010] [08:00:14] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [08:00:18] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [08:00:18] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [08:02:12] <guido_g> no, you have to do it yourself
[Wednesday 22 September 2010] [08:07:53] <raz> hm
[Wednesday 22 September 2010] [08:08:36] <raz> the taskvent example doesn’t seem to inspect the return value at all
[Wednesday 22 September 2010] [08:08:53] <raz> would s_send return an error if no worker picks the message up?
[Wednesday 22 September 2010] [08:09:27] * raz tries to find some semantic overview of the different modes/types
[Wednesday 22 September 2010] [08:10:37] <raz> the zmq_send man-page does not mention such an error tho
[Wednesday 22 September 2010] [08:11:19] <guido_g> there should be a description what happens for the socket types if something goes wrong
[Wednesday 22 September 2010] [08:11:51] * guido_g needs an new english grammar module though
[Wednesday 22 September 2010] [08:12:20] <raz> hmyea
[Wednesday 22 September 2010] [08:12:30] <raz> the ZMQ_PUSH section doesn’t tell me what happens when a msg can not be delivered tho
[Wednesday 22 September 2010] [08:12:41] <raz> or rather.. i assumed there was some sort of acknowledgement going on
[Wednesday 22 September 2010] [08:12:45] <guido_g> nothing
[Wednesday 22 September 2010] [08:12:46] <raz> which is apparently on the case
[Wednesday 22 September 2010] [08:13:19] <raz> s/on/not/
[Wednesday 22 September 2010] [08:14:02] <guido_g> there is no guarantee for delivery in mq
[Wednesday 22 September 2010] [08:15:03] <raz> hmm
[Wednesday 22 September 2010] [08:30:56] Quit xla has left this server (Quit: leaving).
[Wednesday 22 September 2010] [08:37:26] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 22 September 2010] [08:37:48] <ajsie> zeromq doesnt support persistence?
[Wednesday 22 September 2010] [08:50:47] Quit xla has left this server (Quit: leaving).
[Wednesday 22 September 2010] [08:57:32] Quit sbahra has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [09:15:01] <guido_g> it can store messages, but that’s not what you know from broker based solutions
[Wednesday 22 September 2010] [09:23:58] Quit zsxxsz has left this server (Quit: Page closed).
[Wednesday 22 September 2010] [09:28:49] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 22 September 2010] [09:32:55] <keffo> mm, nice.. code dated 3/11/78
[Wednesday 22 September 2010] [09:37:24] <vanadium> If I have one push sockets sending workloads to a number of threads with pull sockets, the work will be evenly distributed over all threads even if some of them take way longer per workload, right?
[Wednesday 22 September 2010] [09:37:38] <vanadium> It is just a round-robin thing and not actually communicating back who is ready for more work or anything?
[Wednesday 22 September 2010] [09:50:11] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 22 September 2010] [09:50:42] <drbobbeaty> vanadium: I’ve not used the PUSH/PULL sockets before, but I believe that you are correct in your assumption. You have a single socket (PUSH) that’s bound to some URL, and a series of “worker threads” that each has a socket (PULL) that’s connected to that same URL. Then, as the single “producer” socket sends out messages, they are picked up and worked on, in turn, by the worker threads.
[Wednesday 22 September 2010] [09:51:24] <drbobbeaty> As i understand it, there are several ways to handle the distribution - round-robin, least-recently-used, etc., but I’m not as familiar with those concepts or how to set them up.
[Wednesday 22 September 2010] [09:51:24] <vanadium> Thank you
[Wednesday 22 September 2010] [09:51:58] <drbobbeaty> no problem
[Wednesday 22 September 2010] [09:58:09] Part ajsie has left this channel.
[Wednesday 22 September 2010] [10:03:28] <guido_g> re
[Wednesday 22 September 2010] [10:13:17] <raz> has anyone used zmq with ruby eventmachine?
[Wednesday 22 September 2010] [10:13:20] <raz> does it blend?
[Wednesday 22 September 2010] [10:17:39] <cremes> raz: the 2.0.x branch doesn’t blend well with EM; the 2.1.x branch might
[Wednesday 22 September 2010] [10:18:10] <raz> well, i have an existing em app that i’d like to add zmq to
[Wednesday 22 September 2010] [10:18:30] <raz> zmqmachine looks like it’s meant to be a replacement for em
[Wednesday 22 September 2010] [10:18:41] <cremes> right
[Wednesday 22 September 2010] [10:19:01] <cremes> you’d have to wire the 0mq sockets into the EM reactor loop
[Wednesday 22 September 2010] [10:19:09] <cremes> that isn’t possible with any 0mq release right now
[Wednesday 22 September 2010] [10:19:25] <raz> hrm
[Wednesday 22 September 2010] [10:19:28] <cremes> the 2.1.x branch has a few new file descriptor elements exposed which will make it feasible to do so
[Wednesday 22 September 2010] [10:19:43] <raz> any dirty trick possible with a thread or so?
[Wednesday 22 September 2010] [10:19:55] <cremes> of course, but that’s cheating :)
[Wednesday 22 September 2010] [10:19:59] <raz> hehe
[Wednesday 22 September 2010] [10:20:11] <raz> i don’t need it super pretty or fast, but super reliable would be nice ;)
[Wednesday 22 September 2010] [10:21:05] <raz> sadly my eventmachine fu isn’t all that advanced, not sure if i’d get the dispatching between eventmachine and a separate thread right
[Wednesday 22 September 2010] [10:21:06] <cremes> you would need to run the 0mq stuff in a thread separate from EM and any interaction would need to be scheduled via EM.schedule
[Wednesday 22 September 2010] [10:21:23] <cremes> 0mq -> EM would be via EM.schedule
[Wednesday 22 September 2010] [10:21:33] <cremes> EM -> 0mq could be handled straight up
[Wednesday 22 September 2010] [10:22:04] <raz> except when the 0mq call blocks (in the latter case)
[Wednesday 22 September 2010] [10:22:16] <raz> guess i should use a queue both ways
[Wednesday 22 September 2010] [10:22:23] <cremes> raz: you’d have to pass ZM_NOBLOCK to all send/recv calls
[Wednesday 22 September 2010] [10:22:48] <raz> hmm
[Wednesday 22 September 2010] [10:23:15] <cremes> you could maybe run zmqmachine and EM in parallel; all of that NO_BLOCK stuff is handled for you then
[Wednesday 22 September 2010] [10:23:18] <raz> sounds like that may be worth a shot
[Wednesday 22 September 2010] [10:23:25] <cremes> let us know how it turns out
[Wednesday 22 September 2010] [11:50:05] Quit dbudworth has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [11:53:39] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 22 September 2010] [12:48:09] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [12:54:38] Quit plol has left this server (Ping timeout: 264 seconds).
[Wednesday 22 September 2010] [12:57:31] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 22 September 2010] [13:35:07] Part dbudworth has left this channel.
[Wednesday 22 September 2010] [13:35:39] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 22 September 2010] [13:36:58] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [13:45:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [13:45:56] Quit plq has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [13:49:42] <mankins> hi–i’ve got an installation problem with 2.0.9 under linux. configure: error: cannot link with -luuid, install uuid-dev.
[Wednesday 22 September 2010] [13:49:58] <mankins> only I’ve already got uuid-devel installed (under fedora core 8 on ec2)
[Wednesday 22 September 2010] [13:50:39] <mankins> libuuid.so.1 is at /lib ... not sure where to go next. Does this ring any bells?
[Wednesday 22 September 2010] [13:54:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [13:56:58] * mankins looks to mirror: oh! ln -s /usr/lib/libuuid.so.1 /usr/lib/libuuid.so fixed it.
[Wednesday 22 September 2010] [14:14:59] * mankins oh, maybe e2fsprogs-devel as well.
[Wednesday 22 September 2010] [14:18:21] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [14:34:10] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [14:39:22] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [14:44:52] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [15:02:18] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 22 September 2010] [16:05:41] Quit xla has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [16:08:45] Quit
antares_ has left this server (Quit:
antares_).
[Wednesday 22 September 2010] [16:29:17] <CIA-20> jzmq: 03Stefan Majer 07master * rf3aa751 10/ (jzmq.spec pom.xml): RPM Packaging and Mavenized, first step. -
http://bit.ly/a6fxhX[Wednesday 22 September 2010] [17:07:10] <cremes> i’m wondering if anyone has seen an assertion failure in 2.0.9
[Wednesday 22 September 2010] [17:07:11] <cremes> Assertion failed: !engine (session.cpp:287)
[Wednesday 22 September 2010] [17:07:49] <cremes> i’m hitting that a lot but i can’t figure out why; i’m having a hard time reducing the code to a small example
[Wednesday 22 September 2010] [17:11:54] <pieterh> cremes: that’s a reported issue afair
[Wednesday 22 September 2010] [17:12:11] <pieterh> it’s caused by... /me vaguely recalls... identity crisis
[Wednesday 22 September 2010] [17:12:23] * pieterh goes to check issue tracker
[Wednesday 22 September 2010] [17:12:27] <cremes> ah... 2+ sockets with same identity...?
[Wednesday 22 September 2010] [17:12:37] <pieterh> yeah
[Wednesday 22 September 2010] [17:12:45] <pieterh> evil twin syndrome
[Wednesday 22 September 2010] [17:12:47] <cremes> cool, then i know where my bug is.... :)
[Wednesday 22 September 2010] [17:13:00] <pieterh> issue 30
[Wednesday 22 September 2010] [17:13:23] <pieterh> hey chuck, Chicago rocks :-)
[Wednesday 22 September 2010] [17:13:59] <cremes> i’m glad you had a good time... may i assume the trip was worthwhile from both a pleasure and business perspective?
[Wednesday 22 September 2010] [17:14:43] * pieterh is typing with his 1-month son in his arms... slightly tricky
[Wednesday 22 September 2010] [17:14:49] <cremes> heh
[Wednesday 22 September 2010] [17:15:07] <pieterh> 0mq + chicago = boom!
[Wednesday 22 September 2010] [17:15:21] <cremes> good boom or bad boom? ;)
[Wednesday 22 September 2010] [17:15:32] <pieterh> the good kind :-)
[Wednesday 22 September 2010] [17:15:50] <cremes> great... next time you come out bring the team
[Wednesday 22 September 2010] [17:16:24] <cremes> oops, gotta run; catch ya later!
[Wednesday 22 September 2010] [17:17:32] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 22 September 2010] [17:17:56] <pieterh> cyl
[Wednesday 22 September 2010] [17:21:10] <mato> pieterh: yo
[Wednesday 22 September 2010] [17:21:18] <mato> pieterh: just got back from dinner, etc with sustrik
[Wednesday 22 September 2010] [17:21:38] <mato> pieterh: linux-kongress starts tomorrow, martin is speaking around 5pm
[Wednesday 22 September 2010] [17:24:00] <pieterh> hey mato :-)
[Wednesday 22 September 2010] [17:24:11] <pieterh> how’s nurnberg?
[Wednesday 22 September 2010] [17:24:21] <mato> nice
[Wednesday 22 September 2010] [17:24:27] <mato> good beer
[Wednesday 22 September 2010] [17:24:35] <mato> some kind of local beer fest is on
[Wednesday 22 September 2010] [17:24:39] <pieterh> in germany? lol...
[Wednesday 22 September 2010] [17:25:20] <pieterh> there’s some kind of wine festival in bratislava next weekend
[Wednesday 22 September 2010] [17:25:49] <pieterh> hey, mato, random question about messages
[Wednesday 22 September 2010] [17:26:08] <pieterh> i vaguely recall El Sustrik saying that zmq_send nullifies the message it sends
[Wednesday 22 September 2010] [17:26:24] <pieterh> even insisting this should be clearly explained in the guide
[Wednesday 22 September 2010] [17:26:33] <mato> zmq_recv does
[Wednesday 22 September 2010] [17:26:39] <mato> send doesn’t afaik
[Wednesday 22 September 2010] [17:26:40] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 22 September 2010] [17:26:47] <pieterh> ? recv nullifies the message?
[Wednesday 22 September 2010] [17:26:56] * pieterh needs another glass of vino to understand this
[Wednesday 22 September 2010] [17:27:15] <pieterh> ah, before receiving, previous contents are discarded...
[Wednesday 22 September 2010] [17:28:25] <pieterh> mato: nah, it was send
[Wednesday 22 September 2010] [17:28:32] <pieterh> i found the line in the userguide:
[Wednesday 22 September 2010] [17:28:33] <pieterh> Note than when you have passed a message to zmq_send[3], 0MQ will clear the message, i.e. set the data to empty. You cannot send the same message twice, and you cannot access the message data after sending it.
[Wednesday 22 September 2010] [17:28:44] <pieterh> yet this is wrong
[Wednesday 22 September 2010] [17:28:53] Quit plol has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [17:29:15] <pieterh> could you bounce this off Sustrik?
[Wednesday 22 September 2010] [17:30:04] <mato> that’s wrong
[Wednesday 22 September 2010] [17:30:12] <mato> of course you can send a message multiple times
[Wednesday 22 September 2010] [17:30:15] <mato> i do it all the time
[Wednesday 22 September 2010] [17:30:18] <starkdg> i thought zmq_msg_close() is what kills messages ?
[Wednesday 22 September 2010] [17:30:24] <mato> yes, precisely
[Wednesday 22 September 2010] [17:30:42] <mato> recv is special in that it destroys the content of the message being received into (if any)
[Wednesday 22 September 2010] [17:30:46] <mato> which makes sense
[Wednesday 22 September 2010] [17:30:50] <mato> send does not touch it
[Wednesday 22 September 2010] [17:30:51] <pieterh> mato: that all makes sense
[Wednesday 22 September 2010] [17:31:13] <pieterh> yet I wrote that after sustrik explicitly asked me to emphasize that
[Wednesday 22 September 2010] [17:31:20] <pieterh> maybe he meant recv not send...
[Wednesday 22 September 2010] [17:31:25] <pieterh> or i noted it wrong
[Wednesday 22 September 2010] [17:31:32] <mato> probably
[Wednesday 22 September 2010] [17:31:47] <pieterh> send is 100% safe?
[Wednesday 22 September 2010] [17:31:57] <mato> it’s quite obviously wrong, i have lots of code that sends the same msg_t X times
[Wednesday 22 September 2010] [17:32:00] <pieterh> kurwa
[Wednesday 22 September 2010] [17:32:03] <pieterh> ok, thanks...
[Wednesday 22 September 2010] [17:32:04] <mato> :-)
[Wednesday 22 September 2010] [17:32:58] <starkdg> you can recieve and send the same message, right ? ive been doing that
[Wednesday 22 September 2010] [17:33:49] <mato> yes
[Wednesday 22 September 2010] [17:58:24] Quit mankins has left this server (Quit: mankins).
[Wednesday 22 September 2010] [18:01:59] Quit ModusPwnens has left this server.
*** on Wed Sep 22 18:31:08 2010
[Wednesday 22 September 2010] [18:31:08] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [18:31:12] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [18:31:12] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [18:31:28] Quit travlr has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [18:47:50] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 22 September 2010] [18:50:43] Quit starkdg has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [18:58:33] Part jashmenn has left this channel.
[Wednesday 22 September 2010] [19:01:12] Nick MJW is now known as mjw9100.
[Wednesday 22 September 2010] [19:08:43] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [19:18:53] Quit starkdg has left this server (Ping timeout: 245 seconds).
[Wednesday 22 September 2010] [19:21:12] Part mjw9100 has left this channel.
[Wednesday 22 September 2010] [19:24:04] Part mjw9100 has left this channel.
[Wednesday 22 September 2010] [19:25:33] Quit friendshi has left this server (Quit: friendshi).
[Wednesday 22 September 2010] [19:42:25] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [19:42:38] Quit pieterh has left this server (Ping timeout: 245 seconds).
[Wednesday 22 September 2010] [19:54:03] Part mikejs has left this channel.
[Wednesday 22 September 2010] [20:56:45] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Wednesday 22 September 2010] [21:03:03] Quit andrewvc has left this server (Quit: andrewvc).
[Wednesday 22 September 2010] [21:59:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [22:40:39] Quit friendshi has left this server (Client Quit).
[Wednesday 22 September 2010] [22:45:48] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 22 September 2010] [23:03:12] Quit grahamalot has left this server (Quit: grahamalot).
[Wednesday 22 September 2010] [23:46:30] Quit yrashk has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [00:28:54] <Pe_Ell> anyone know if there is a roadmap for release dates? Just wondering since I didn’t notice one on github or the site off hand...
[Thursday 23 September 2010] [01:11:17] Quit BooTheHamster1 has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:11:37] Join BooTheHamster1 has joined this channel (
~boo@94.25.51.179).
[Thursday 23 September 2010] [01:24:51] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:27:37] Quit BooTheHamster1 has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:39:53] <guido_g> nope, there is no such thing
[Thursday 23 September 2010] [01:40:21] <guido_g> standard quote is “it’ll be released when its ready” :)
[Thursday 23 September 2010] [01:42:47] <Pe_Ell> heh, k
[Thursday 23 September 2010] [01:43:23] <Pe_Ell> there’s a feature in 2.1 that I’m waiting for... wasn’t sure how stable the git branch was... guess I’ll just keep on waiting... :)
[Thursday 23 September 2010] [01:43:37] <guido_g> if you’re around longer, ask pieterh, mato or sustrik, they do know the details
[Thursday 23 September 2010] [01:44:20] <Pe_Ell> k. I’ll just leave this open for now while I read more code/docs...
[Thursday 23 September 2010] [01:55:59] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [01:56:47] Quit plol has left this server (Ping timeout: 240 seconds).
[Thursday 23 September 2010] [01:59:54] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [02:13:01] Quit raz has left this server (Quit: fump).
[Thursday 23 September 2010] [02:58:15] Quit andrewvc has left this server (Quit: andrewvc).
[Thursday 23 September 2010] [03:19:36] Quit plol has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [03:42:56] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [04:36:10] Nick
mikko__ is now known as mikko.
[Thursday 23 September 2010] [05:00:00] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 23 September 2010] [05:25:10] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 23 September 2010] [05:49:09] Quit jugg has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [06:42:11] Join plq has joined this channel (
~plq@78.189.30.218).
[Thursday 23 September 2010] [08:25:06] Part mjw9100 has left this channel.
[Thursday 23 September 2010] [08:53:57] <pieter_hintjens> hi folk, it’s quiet today
[Thursday 23 September 2010] [08:54:21] <pieter_hintjens>
*folks
[Thursday 23 September 2010] [08:54:32] <pieter_hintjens> I’ve pushed out a new version of the user guide
[Thursday 23 September 2010] [09:01:50] Quit mytrile has left this server (Ping timeout: 252 seconds).
[Thursday 23 September 2010] [09:28:17] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [09:55:49] Quit sbahra has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [09:56:49] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [10:01:27] Quit plol has left this server (Read error: Operation timed out).
[Thursday 23 September 2010] [10:18:19] Quit plq has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [10:39:02] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Thursday 23 September 2010] [10:58:13] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [11:19:34] <friendshi> hi, I’m having an issue when trying to do non-block recv... I always immediately receive an exception ‘Resource temporarily unavailable’, anyone know where i might be going wrong?
[Thursday 23 September 2010] [11:20:30] <guido_g> maybe the connect wasn’t done at this point in time
[Thursday 23 September 2010] [11:20:52] <friendshi> hmm it works fine with i do blocking, and i have tried sleeping for 5+ seconds between connect and recv
[Thursday 23 September 2010] [11:21:13] <guido_g> because the real connection handling is in a seperate thread, you might do the receive while the underlying connection is not yet established
[Thursday 23 September 2010] [11:23:33] <friendshi> i dont think that can be it... when i do it without block it begins reading my messages immediately... unless i am misunderstanding what you are suggesting
[Thursday 23 September 2010] [11:23:42] <friendshi>
*without nonblock
[Thursday 23 September 2010] [11:24:04] <guido_g> i didn’t suggest something
[Thursday 23 September 2010] [11:24:19] <friendshi> sorry i just mean, what you are saying the issue could be
[Thursday 23 September 2010] [11:24:40] <guido_g> one idea is to make the first receive blocking and all others not
[Thursday 23 September 2010] [11:25:01] <guido_g> if this works, it’s time for an issue i’d say
[Thursday 23 September 2010] [11:26:26] * guido_g checks the docu
[Thursday 23 September 2010] [11:26:27] <friendshi> same issue :(
[Thursday 23 September 2010] [11:26:45] <friendshi> it does receive first message fine, then immediate fail on nonblock recv
[Thursday 23 September 2010] [11:26:53] <guido_g> then try to make a small program that shows the problem and show the code
[Thursday 23 September 2010] [11:27:08] <friendshi> sure
[Thursday 23 September 2010] [11:45:32] Nick rbancrof1 is now known as rbancroft.
[Thursday 23 September 2010] [11:45:52] <vagn> greetings. Is there an example somewhere on how to shutdown a ZMQ_REP socket so the otherside doesn’t hang when the service returns?
[Thursday 23 September 2010] [11:46:30] Join ChicagoBud has joined this channel (
~bud@208.78.39.48).
[Thursday 23 September 2010] [11:46:57] <vagn> I want to stop getting requests, but finish all the replies, then exit the program.
[Thursday 23 September 2010] [11:47:48] <vagn> but the requests are queued, right? so the other side things it should get a reply. so basically I have to shutdown the incoming queue, too.
[Thursday 23 September 2010] [11:47:57] <vagn>
*thinks
[Thursday 23 September 2010] [11:53:19] <guido_g> i’d do this kind of synchronization with explicit messages, much easier imho
[Thursday 23 September 2010] [11:54:45] <vagn> thinking some more... that still doesn’t cure tha case where the server crashes.
[Thursday 23 September 2010] [11:56:02] <vagn> maybe I should start over and do things with pub/sub or multicast. I don’t care if I lose a message. the next one that comes along is good enough.
[Thursday 23 September 2010] [12:14:43] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 23 September 2010] [12:14:56] Quit
kayiwa_ has left this server (Quit:
kayiwa_).
[Thursday 23 September 2010] [12:16:33] <friendshi> i find that my problem was that in pyzmq document/source it states ‘If NOBLOCK is set, this method will return None if a message is not ready’ but that is untrue, I believe it is throwing exception is no message exist
[Thursday 23 September 2010] [12:16:55] <friendshi> maybe i am misreading or misunderstanding though
[Thursday 23 September 2010] [12:22:21] <keffo> oh by my might beard! I hate 3g
[Thursday 23 September 2010] [12:37:51] Quit xla has left this server (Quit: leaving).
[Thursday 23 September 2010] [12:58:56] <bgranger> friendshi: hi
[Thursday 23 September 2010] [13:19:51] Quit vagn has left this server (Quit: Ex-Chat).
[Thursday 23 September 2010] [14:56:38] Quit mytrile has left this server (Ping timeout: 272 seconds).
[Thursday 23 September 2010] [15:04:07] Quit miguelito has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [15:39:01] Notice -Martinp23- :[Global Notice] Hi folks. One of our servers (jordan) needs to undergo emergency maintenance in the next 30 minutes. The maintenance window will be 48 hours or so. The result of this will be a netsplit as 3000 users, who have already been notified seperately, disconnect. I’m sorry for the inconvenience. If you’ve any questions, pop into #freenode. Thanks!
[Thursday 23 September 2010] [15:40:44] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [16:00:40] Quit miguelito has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:40] Quit yrashk has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:41] Quit mato has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:41] Quit jond has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:33:01] Quit plol has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [16:42:38] Quit xla has left this server (Ping timeout: 264 seconds).
[Thursday 23 September 2010] [16:44:07] Quit halfie has left this server (Quit: Leaving).
[Thursday 23 September 2010] [17:01:17] Quit swapi has left this server (Quit: Page closed).
[Thursday 23 September 2010] [17:13:05] Quit bgranger has left this server (Read error: Operation timed out).
[Thursday 23 September 2010] [17:22:48] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 23 September 2010] [17:23:03] Quit miguelito has left this server (Ping timeout: 245 seconds).
[Thursday 23 September 2010] [18:01:00] Quit andrewvc has left this server (Quit: andrewvc).
[Thursday 23 September 2010] [18:15:57] Quit friendshi has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [19:35:43] Part vanadium has left this channel.
[Thursday 23 September 2010] [19:37:44] Quit
mw_ has left this server (Quit:
mw_).
[Thursday 23 September 2010] [19:39:25] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Thursday 23 September 2010] [19:47:22] Quit
mw_ has left this server (Quit:
mw_).
[Thursday 23 September 2010] [20:43:27] Quit
miguelito_ has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [20:49:45] Quit iFire has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [20:55:26] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [21:21:38] <jhawk28> does any other protocols than tcp work on Windows?
[Thursday 23 September 2010] [21:50:58] Quit xla has left this server (Ping timeout: 245 seconds).
[Thursday 23 September 2010] [22:28:06] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [22:31:23] Quit jhawk28 has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [22:32:57] Part starkdg has left this channel.
[Thursday 23 September 2010] [23:55:51] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [00:00:08] Quit mjw9100 has left this server (Remote host closed the connection).
[Friday 24 September 2010] [01:38:56] Quit jsimmons has left this server (Quit: Leaving).
[Friday 24 September 2010] [02:24:03] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [02:27:48] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [02:28:13] Quit guido_g has left this server (Quit: Boom...).
[Friday 24 September 2010] [02:33:22] Quit pieter_hintjens has left this server (Ping timeout: 255 seconds).
[Friday 24 September 2010] [03:09:06] Nick Nickname already in use. Trying
travlr__.
[Friday 24 September 2010] [03:09:11] Topic The channel topic is “Welcome!”.
[Friday 24 September 2010] [03:09:12] CTCP Received Version request from frigg.
[Friday 24 September 2010] [03:09:12] Notice -NickServ- travlr__ is not a registered nickname.
[Friday 24 September 2010] [03:09:15] Mode Channel modes: no messages from outside, topic protection
[Friday 24 September 2010] [03:09:15] Created This channel was created on 2010-02-05 10:44.
[Friday 24 September 2010] [03:09:43] Quit
travlr_ has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [03:10:14] Quit Pe_Ell has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [03:19:13] Quit SecretAgent has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [03:20:26] Quit AlexB has left this server (Ping timeout: 264 seconds).
[Friday 24 September 2010] [03:20:40] Join AlexB has joined this channel (
alex@brasetvik.com).
[Friday 24 September 2010] [03:33:43] Quit Pe-Ell has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [03:34:50] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [04:08:28] Quit xla has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [04:59:18] Quit drbobbeaty has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [05:02:02] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [05:07:43] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 24 September 2010] [05:25:26] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 24 September 2010] [05:30:46] Quit guido_g has left this server (Quit: Boom...).
[Friday 24 September 2010] [05:34:07] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [07:27:21] <Zao> Assuming you manage to build it, I can’t see why OpenPGM wouldn’t work.
[Friday 24 September 2010] [07:28:31] Quit keffo has left this server (Quit: Leaving).
[Friday 24 September 2010] [07:32:40] Part
i__ has left this channel.
[Friday 24 September 2010] [07:40:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [08:48:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [08:55:50] Quit plol has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [09:07:09] Part mjw9100 has left this channel.
[Friday 24 September 2010] [09:29:53] Part mjw9100 has left this channel.
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r7073557 10/ src/org/zeromq/ZMQ.java : Javadoc, reduction of the visibility of a lot of flags reduced to private because they are only used inside public setters/getters. Fix of some checkstyle warnings. -
http://bit.ly/aGlNur[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r34d656f 10/ src/org/zeromq/ZMQ.java :
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Deprecation of ZMQ.UPSTREAM and ZMQ.DOWNSTREAM
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Socket.subscribe and Socket.unsubscribe only on a Socket of Type ZMQ.SUB -
http://bit.ly/cQRpmM[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r86512f8 10/ src/org/zeromq/ZMQ.java :
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Revert the subscribe/unsubscribe check,
[Friday 24 September 2010] [09:45:34] <CIA-20> jzmq: Whitespace and identation with zeromq rules -
http://bit.ly/9wrDyQ[Friday 24 September 2010] [09:45:35] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * r3ba6cc1 10/ src/org/zeromq/ZMQ.java : Pulled changes from Stefan Majer to improve documentation and visibility. Made minor modifications. -
http://bit.ly/awIGSC[Friday 24 September 2010] [09:55:50] Part mjw9100 has left this channel.
[Friday 24 September 2010] [10:53:29] <codebeaker> hey, quick sanity check - can I (should I?) include the zeromq repository in my own project as a git submodule ?
[Friday 24 September 2010] [11:05:00] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [11:10:43] Quit rhi has left this server (Client Quit).
[Friday 24 September 2010] [11:15:43] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [11:16:58] <cremes> codebeaker: that’s a good question for some git experts... i don’ t think there are any in this channel
[Friday 24 September 2010] [11:18:04] <codebeaker> cremes: I don’t need a git expert, my question was more about whether it makes sense, if you distribute a closed source application, between a couple of devlopers (mac and windows targets) how are other people typically including the mq sources in their projects
[Friday 24 September 2010] [11:18:46] <codebeaker> I noticed there were separate downloads available for Windows and Unix, so that would lead me to believe you have to build them differently
[Friday 24 September 2010] [11:20:51] <cremes> i don’t know how to answer your question... hopefully someone else will come along who can
[Friday 24 September 2010] [11:21:09] <codebeaker> thanks cremes - appreciate the attention anyway
[Friday 24 September 2010] [11:21:33] <codebeaker> basically, I could include the whole repository from mq in my own, and then specify build flags - but I’m not sure how smart that would be
[Friday 24 September 2010] [11:31:29] <pieter_hintjens> codebeaker: including 0MQ as a git submodule would be a pretty good solution, yes
[Friday 24 September 2010] [11:32:09] <pieter_hintjens> at least until 0MQ becomes a standard package on the various distros
[Friday 24 September 2010] [11:32:29] <pieter_hintjens> and definitely a good idea if you want specific versions or branches of 0MQ
[Friday 24 September 2010] [11:33:08] <pieter_hintjens> but the distribution packages are identical except for line endings and compression format
[Friday 24 September 2010] [11:35:49] <codebeaker> pieter_hintjens: ^ thanks that’s exactly the info I needed :-D
[Friday 24 September 2010] [11:35:58] <codebeaker> bew, mq is freakin awesome
[Friday 24 September 2010] [11:36:01] <codebeaker> btw*
[Friday 24 September 2010] [11:36:02] Quit codebeaker has left this server (Quit: codebeaker).
[Friday 24 September 2010] [11:36:04] <pieter_hintjens> codebeaker: hopefully it’s also accurate :-)
[Friday 24 September 2010] [12:00:13] Quit plol has left this server (Ping timeout: 252 seconds).
[Friday 24 September 2010] [12:12:10] <mjw9100> folks, just confirming that an “ordinary” fd for zmq_poll() can be an epoll fd?
[Friday 24 September 2010] [12:19:23] <mjw9100> i.e. can I set zmq_pollitem_t.fd to an epoll() fd?
[Friday 24 September 2010] [12:19:56] <mjw9100> according to the epoll() Q&A, these epoll fd’s are selectable, etc.
[Friday 24 September 2010] [12:23:19] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [12:24:19] <jhawk28> hello
[Friday 24 September 2010] [12:24:59] <jhawk28> having a problem moving the compiled version of the windows libraries to another machine
[Friday 24 September 2010] [12:55:47] <praveenbm5> hello all
[Friday 24 September 2010] [12:56:03] <praveenbm5> i hope i am in the right room to discuss ZeroMQ
[Friday 24 September 2010] [12:56:12] Quit omarkj has left this server (Quit: omarkj).
[Friday 24 September 2010] [12:56:42] <codebeaker> praveenbm5: the name says it all!
[Friday 24 September 2010] [12:56:46] <praveenbm5> is anybody alive here
[Friday 24 September 2010] [12:57:15] <praveenbm5> i was looking on the website regarding some guidance using zmq_poll
[Friday 24 September 2010] [12:57:53] <praveenbm5> i just posted a comment there
[Friday 24 September 2010] [13:03:35] <codebeaker> ^ I’m inqo mq for about 5 minutes now, what’s the behaviour if you broadcast a message to an upstream (is the terminology the right way round there ) - but nothing is listening there yet, is it lost ?
[Friday 24 September 2010] [13:04:58] <praveenbm5> its not lost
[Friday 24 September 2010] [13:05:25] <praveenbm5> queues exist even when the end point doesnt yet exist
[Friday 24 September 2010] [13:05:53] <praveenbm5> i suggest you read the guide on zeromq.org
[Friday 24 September 2010] [13:05:53] <codebeaker> ah, ok - great - i’m actually looking into it for writing a plugin for winamp that fires off a list of whats playing
[Friday 24 September 2010] [13:06:14] <codebeaker> (and I can’t be sure the other app is running)
[Friday 24 September 2010] [13:06:52] <praveenbm5> sorry i got your questiong wrong
[Friday 24 September 2010] [13:07:18] <praveenbm5> you can connect to an end point from a zmq socket before the endpoint exists
[Friday 24 September 2010] [13:07:19] <jhawk28> codebeaker: the new terminology for “upstream” is “push”
[Friday 24 September 2010] [13:07:32] <praveenbm5> but regarding messages
[Friday 24 September 2010] [13:07:38] <codebeaker> no problem, winamp loads .dlls like plugins, and I wanted that plugin to shout out (over zmq) “hey, something played in winamp” - what happens if something isn’t listening ?
[Friday 24 September 2010] [13:08:07] <praveenbm5> push is for parallel worker based execution
[Friday 24 September 2010] [13:08:14] <jhawk28> it sits in the local queue till the socket can connect I believe
[Friday 24 September 2010] [13:08:49] <jhawk28> upstream/downstream == push/pull
[Friday 24 September 2010] [13:09:31] <jhawk28> the only one that doesn’t “keep” the message is the pub socket when noone has connected (I believe)
[Friday 24 September 2010] [13:10:15] <codebeaker> jhawk28: great, and that local queue is transient, in my example - if the closed winamp, and the plugin unloaded - it would be lost, right ?
[Friday 24 September 2010] [13:12:04] <jhawk28> yes, I think so
[Friday 24 September 2010] [13:12:29] <codebeaker> jhawk28: thanks, I’m going to screw around in the Ruby console a little - but that sounds like exactly the behaviour I was looking for
[Friday 24 September 2010] [13:12:41] <jhawk28> you would have to manually persist if you dont want it transient
[Friday 24 September 2010] [13:13:05] <codebeaker> yeah, jhawk28 - I’m looking for it not to persist, or at least to know if it was read off (I suppose I could always examine the local queue)
[Friday 24 September 2010] [13:13:06] <jhawk28> easiest way to see is to download one of the samples and then compile it
[Friday 24 September 2010] [13:13:22] <jhawk28> start them both a couple of times, but in different orders
[Friday 24 September 2010] [13:13:27] <codebeaker> also, lsat question (and I can infer a litlte from the guide) the ipc:// dsn specification, it is portable across windows/unix, I assume ?
[Friday 24 September 2010] [13:13:43] <jhawk28> I havent gotten ipc to work on my windows machine
[Friday 24 September 2010] [13:13:48] <jhawk28> the inproc seems to
[Friday 24 September 2010] [13:14:25] <codebeaker> ahh - I was sortof counting on that - however since it’s trivial, and it’s not really IPC, if it’s two separate apps a .dll in winamp, and something else standalone - maybe IPC is the wrong paradigm now
[Friday 24 September 2010] [13:14:35] <jhawk28> from the man page “The inter-process transport passes messages between local processes using a system-dependent IPC mechanism.”
[Friday 24 September 2010] [13:15:12] <jhawk28> vs “The in-process transport passes messages via memory directly between threads sharing a single MQ context”
[Friday 24 September 2010] [13:15:51] <codebeaker> ah, so IPC is certainly somehow better suited, but a socket would work just as well
[Friday 24 September 2010] [13:18:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [13:19:03] <jhawk28> they are both “sockets”, but the IPC is dependent on the OS
[Friday 24 September 2010] [13:19:08] <codebeaker> ah, ok
[Friday 24 September 2010] [13:19:20] <jhawk28> so if you want cross platform, you probably need inproc
[Friday 24 September 2010] [13:19:30] <guido_g> no
[Friday 24 September 2010] [13:19:37] <codebeaker> inproc wouldn’t work for talking from a winamp process to another process all together
[Friday 24 September 2010] [13:19:44] <guido_g> ipc is between different processes
[Friday 24 September 2010] [13:20:02] <guido_g> on windows, where ipc is not available, use
tcp://127.0.0.1[Friday 24 September 2010] [13:20:03] <codebeaker> I assume I need to go out-of-band if IPC doesn’t work on Windows, and use a real socket, like TCP or sth. that windows supports
[Friday 24 September 2010] [13:20:14] Quit titto has left this server (Quit: Ex-Chat).
[Friday 24 September 2010] [13:20:16] <guido_g> ipc is a real socket
[Friday 24 September 2010] [13:20:26] <codebeaker> ah, but a unix file socket, right ?
[Friday 24 September 2010] [13:20:31] <guido_g> right
[Friday 24 September 2010] [13:20:47] <codebeaker> ok, I assumed (foolishly) that ipc:// on windows would fall-back to tcp://
[Friday 24 September 2010] [13:22:41] <jhawk28> so, inproc is between threads, but not between processes
[Friday 24 September 2010] [13:22:45] <jhawk28> ?
[Friday 24 September 2010] [13:22:52] <codebeaker> jhawk28: certainly sounds like that
[Friday 24 September 2010] [13:24:35] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [13:27:44] <jhawk28> has anyone done the zmq and jzmq build on vs2010 on Win7_64bit?
[Friday 24 September 2010] [13:28:34] <guido_g> have a look at the mailing list, it’s been discussed there afair
[Friday 24 September 2010] [13:38:01] Quit keffo has left this server (Ping timeout: 255 seconds).
[Friday 24 September 2010] [13:41:27] Part mjw9100 has left this channel.
[Friday 24 September 2010] [13:48:02] Quit keffo has left this server (Ping timeout: 264 seconds).
[Friday 24 September 2010] [13:53:11] <jond> codebreaker: ipc isnt implemented on windows, inproc is between threads.
[Friday 24 September 2010] [13:53:20] Part praveenbm5 has left this channel.
[Friday 24 September 2010] [14:01:12] <codebeaker> jond: rgr that (from earlier!) - pretty sure i’ll go with a tcp socket on both platforms
[Friday 24 September 2010] [14:01:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:14:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:23:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:33:26] Quit mytrile has left this server (Quit: Leaving.).
[Friday 24 September 2010] [14:38:47] <jhawk28> hurrah! I got VS2010 to compile zmq and jzmq
[Friday 24 September 2010] [14:38:55] <codebeaker> w00t, well done :-D
[Friday 24 September 2010] [14:48:17] Quit mytrile has left this server (Read error: Operation timed out).
[Friday 24 September 2010] [14:53:02] Quit sustrik has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [15:01:35] Quit xla has left this server (Quit: leaving).
[Friday 24 September 2010] [15:18:40] <rgl> hi
[Friday 24 September 2010] [15:23:09] <codebeaker> hi rgl
[Friday 24 September 2010] [15:33:27] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [15:45:33] Quit starkdg has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [15:47:10] Part mjw9100 has left this channel.
[Friday 24 September 2010] [16:08:53] Quit
mw_ has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [16:18:51] <jond> codebeaker: ipc is faster on linux, than tcp loopback; on solaris it made no difference as solaris loopback is optimised
[Friday 24 September 2010] [16:19:29] <jond> sustrik: how did the presentation go?
[Friday 24 September 2010] [16:55:58] Quit jhawk28 has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [16:56:36] Quit keffo has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [17:06:00] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [17:25:24] Quit codebeaker has left this server (Quit: codebeaker).
[Friday 24 September 2010] [17:26:15] Quit codebeaker has left this server (Client Quit).
[Friday 24 September 2010] [17:27:25] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 24 September 2010] [17:34:07] Quit jhawk28 has left this server (Remote host closed the connection).
[Friday 24 September 2010] [17:40:26] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [17:48:18] Quit ysynopsis has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [18:24:15] Quit rgl has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [18:44:37] Nick _rgl is now known as rgl.
[Friday 24 September 2010] [19:00:43] Quit bennymack has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [19:19:18] Quit mytrile has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [19:20:18] Quit rgl has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [19:20:40] Quit friendshi has left this server (Remote host closed the connection).
[Friday 24 September 2010] [20:47:31] Quit starkdg has left this server (Quit: goodbye).
[Friday 24 September 2010] [21:56:52] Quit iFire has left this server (Read error: Connection reset by peer).
*** on Fri Sep 24 22:16:23 2010
[Friday 24 September 2010] [22:16:23] Topic The channel topic is “Welcome!”.
[Friday 24 September 2010] [22:16:27] Mode Channel modes: no messages from outside, topic protection
[Friday 24 September 2010] [22:16:27] Created This channel was created on 2010-02-05 10:44.
[Friday 24 September 2010] [22:30:23] Quit
mw_ has left this server (Quit:
mw_).
[Friday 24 September 2010] [22:31:40] Quit grahamalot has left this server (Quit: grahamalot).
[Friday 24 September 2010] [23:09:34] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [23:10:42] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [00:05:07] Quit RobertLJ has left this server (Ping timeout: 252 seconds).
[Saturday 25 September 2010] [01:20:48] Quit rgl has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [01:52:35] Quit lluad has left this server (Quit: lluad).
[Saturday 25 September 2010] [02:32:42] <sustrik> Pe_Ell: 2.1 looks more or less stable already. Please, do use it. If any bug is found, let me know.
[Saturday 25 September 2010] [02:36:19] Quit bgranger has left this server (Quit: bgranger).
[Saturday 25 September 2010] [02:50:20] <sustrik> jond: it went well, i’ll post my paper shortly
[Saturday 25 September 2010] [02:50:39] <sustrik> met a lot of linux kernel folks there
[Saturday 25 September 2010] [02:58:35] <Pe_Ell> wow... chapter3 just doubled in size..
[Saturday 25 September 2010] [02:58:51] Quit rgl has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [03:09:18] Quit keffo has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [03:25:07] <Pe_Ell> sustrik, cool. Mostly was interested in accessing the “socket” more directly so there could be some attempt of creting somelike like a Perl AnyEvent wrapper...
[Saturday 25 September 2010] [03:26:37] <Pe_Ell> wow... my english sucks...
[Saturday 25 September 2010] [03:29:47] <sustrik> Pe_Ell: yes, feel free to play with that
[Saturday 25 September 2010] [03:30:09] <sustrik> there’s ZMQ_FD socket option that allows you to do that
[Saturday 25 September 2010] [03:30:33] <Pe_Ell> yeah I was looking over the code and currently reading the guide...
[Saturday 25 September 2010] [03:30:40] <Pe_Ell> pretty entertaining and useful
[Saturday 25 September 2010] [03:30:44] <sustrik> however, it’s a bit more complex than using standard 0mq API
[Saturday 25 September 2010] [03:31:03] <sustrik> you have to keep in mind that:
[Saturday 25 September 2010] [03:31:34] <sustrik> 1. ZMQ_FD can signal even if there’s no message available (on internal 0MQ events such as “connection established”)
[Saturday 25 September 2010] [03:31:47] <sustrik> 2. ZMQ_FD is edge triggered
[Saturday 25 September 2010] [03:32:28] <sustrik> if you don’t care about that, it should work just fine
[Saturday 25 September 2010] [03:32:51] <Pe_Ell> yeah I figured I’d finish the guide before digging into the 2.1 code more...
[Saturday 25 September 2010] [04:09:07] Quit mytrile has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [05:25:29] <pieter_hintjens> Pe_Ell: I moved a lot of Ch3 into Ch2 as well
[Saturday 25 September 2010] [05:41:48] Quit mytrile has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [06:34:27] Quit mytrile has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [07:49:04] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [08:12:25] Quit jsimmons has left this server (Ping timeout: 252 seconds).
[Saturday 25 September 2010] [08:13:18] Quit rossij has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [08:34:48] Quit ysynopsis has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [08:58:48] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [09:33:13] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [09:54:24] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [09:57:04] Quit ysynopsis has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [10:41:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [11:46:14] Quit lluad has left this server (Ping timeout: 264 seconds).
[Saturday 25 September 2010] [11:46:14] Nick
lluad_ is now known as lluad.
*** on Sat Sep 25 13:29:53 2010
[Saturday 25 September 2010] [13:29:53] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [13:29:57] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [13:29:57] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [13:43:50] <Pe_Ell> pieter_hintjens: so I should start over? :) I noticed that it had shifted some, I was going to go look at the earlier chapters to see if I’d missed anything... still got a lot of code to read... :)
[Saturday 25 September 2010] [13:44:26] <pieter_hintjens> Pe_Ell: well, most of the new stuff is at the end of Ch2 and then Ch3
[Saturday 25 September 2010] [13:47:05] <Pe_Ell> yeah I figure I’d like a pretty generic AnyEvent wrapper if it’s reasonable to build... I’m still working out the application design for my project though... I just got distracted with new and shiny.... :D
[Saturday 25 September 2010] [13:57:51] <Pe_Ell> so to make sure I’m not retarded... for pubsub I can make one ZMQ_SUB socket and then just set multiple ZMQ_SUBSCRIPT options on it...
[Saturday 25 September 2010] [13:58:24] <Pe_Ell> err, ZMQ_SUBSCRIBE
[Saturday 25 September 2010] [13:58:45] <Pe_Ell> man I don’t even have a spell checker... not sure why I keep writing out the wrong words...
[Saturday 25 September 2010] [13:59:50] <Pe_Ell> ok man page confirmed what I was thinking... and looks like the string is a prefix so that answered my next question about wildcarding...
[Saturday 25 September 2010] [14:36:32] Quit pieter_hintjens has left this server (Ping timeout: 276 seconds).
[Saturday 25 September 2010] [14:55:25] <lluad> Anyone have any pointers for embedding 0mq into another systems event loop - I’m looking at using it with Qt4.
[Saturday 25 September 2010] [15:13:53] Quit lluad has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [15:13:53] Nick
lluad_ is now known as lluad.
[Saturday 25 September 2010] [15:45:01] <cremes> lluad: take a look at the 2.1 branch specifically ZM_FD and ZM_EVENTS
[Saturday 25 September 2010] [15:45:10] <lluad> Excellent, thanks.
*** on Sat Sep 25 17:20:20 2010
[Saturday 25 September 2010] [17:20:20] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [17:20:24] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [17:20:24] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [17:26:46] Quit
antares_ has left this server (Quit:
antares_).
[Saturday 25 September 2010] [17:45:16] Quit
antares_ has left this server (Quit:
antares_).
[Saturday 25 September 2010] [18:09:36] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Saturday 25 September 2010] [18:27:34] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
*** on Sat Sep 25 21:11:14 2010
[Saturday 25 September 2010] [21:11:14] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [21:11:18] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [21:11:18] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [21:48:28] Quit sbahra has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [21:53:53] Quit pasha_bastov has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [22:33:30] Quit lluad has left this server (Quit: lluad).
[Saturday 25 September 2010] [22:59:26] Quit
pasha_bastov_ has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [23:15:34] Join jdroid- has joined this channel (
~user@74.72.57.182).
[Saturday 25 September 2010] [23:16:25] <jdroid-> hello. i’m reading whitepapers:design-v06. am i right that this is mapped to zeromq v 0.6, making it also quite old?
[Saturday 25 September 2010] [23:27:05] Quit iFire has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [23:37:37] Quit jdroid- has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [23:41:07] Quit bgranger has left this server (Quit: bgranger).
[Saturday 25 September 2010] [23:43:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [01:05:48] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 26 September 2010] [01:49:04] Join sbahra has joined this channel (
~sbahra@pool-74*** on Sun Sep 26 12:46:34 2010
[Sunday 26 September 2010] [12:46:34] Topic The channel topic is “Welcome!”.
[Sunday 26 September 2010] [12:46:38] Mode Channel modes: no messages from outside, topic protection
[Sunday 26 September 2010] [12:46:38] Created This channel was created on 2010-02-05 10:44.
[Sunday 26 September 2010] [13:20:48] Quit bennymack has left this server (Ping timeout: 240 seconds).
[Sunday 26 September 2010] [14:50:48] Quit xla has left this server (Ping timeout: 265 seconds).
[Sunday 26 September 2010] [15:10:08] Quit eintr has left this server (Quit: Leaving.).
[Sunday 26 September 2010] [15:16:52] Quit pasha_bastov has left this server (Quit: Leaving).
[Sunday 26 September 2010] [15:43:34] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * rbe79a9f 10/ (4 files): minor beautifications -
http://bit.ly/cH9kWR[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * rcf815e8 10/ (18 files): new interfaces for timers; the implementation is not changed yet -
http://bit.ly/buVKBu[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r8d7bf66 10/ (13 files): common base for all pollers created; the only thing it handles at the moment is ‘load’ -
http://bit.ly/cqLLHi[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r238640a 10/ (15 files): timers properly implemented -
http://bit.ly/9clGnp[Sunday 26 September 2010] [15:54:10] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Sunday 26 September 2010] [16:19:47] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [17:48:57] Quit GeekGod has left this server (Quit: GeekGod).
[Sunday 26 September 2010] [19:18:20] Quit lotrpy has left this server.
[Sunday 26 September 2010] [19:44:00] <jhawk28> hello, anyone here with windows build experience?
[Sunday 26 September 2010] [19:44:00] Quit zmqircd has left this server (Remote host closed the connection).
[Sunday 26 September 2010] [19:44:35] <jhawk28> I’m having trouble getting the windows code to run on other systems that don’t have VS2008 installed
[Sunday 26 September 2010] [20:05:07] <Zao> jhawk28: Several solutions.
[Sunday 26 September 2010] [20:05:49] <Zao> jhawk28: 1) deploy the runtime via installer. 2) deploy the runtime privately in a subdirectory of your application. 3) build your project against the static runtime, with all the downsides and pitfalls that has.
[Sunday 26 September 2010] [20:06:23] <Zao> Deploying privately with 2008 amounts to grabbing the Microsoft.VC90.CRT directory and putting it next to your executable.
[Sunday 26 September 2010] [20:06:36] <Zao> Deploying via installer can be done either via the standalone installer or via merge modules.
[Sunday 26 September 2010] [20:06:47] <jhawk28> I don’t mind installing the vs2008 redist
[Sunday 26 September 2010] [20:06:51] <Zao> Building against static runtime places severe restrictions to multi-module programs.
[Sunday 26 September 2010] [20:07:11] <Zao> Then install the runtime(s) that your application and your dependencies are built against.
[Sunday 26 September 2010] [20:07:19] <jhawk28> but, I can’t figure out how to have zmq compile against the latest version
[Sunday 26 September 2010] [20:07:19] <Zao> If you’re doing things right, that should be a single runtime version.
[Sunday 26 September 2010] [20:08:47] <jhawk28> btw, this is for the java bindings and the zmq library
[Sunday 26 September 2010] [20:10:19] <jhawk28> but it didn’t change the values in the manifest
[Sunday 26 September 2010] [20:10:46] <Zao> I feel sorry for you. Building against and deploying the right 2008 runtime is great fun.
[Sunday 26 September 2010] [20:10:50] <Zao> Especially with the hotfixes.
[Sunday 26 September 2010] [20:11:57] <jhawk28> :)
[Sunday 26 September 2010] [20:12:01] <jhawk28> :(
[Sunday 26 September 2010] [20:14:32] <jhawk28> new to the whole Visual Studio world so its mostly painful right now
[Sunday 26 September 2010] [20:14:56] <jhawk28> mostly happy that ZMQ compiles out of the box
[Sunday 26 September 2010] [20:16:30] <jhawk28> seems to cover your 3 options
[Sunday 26 September 2010] [20:19:09] Quit xla has left this server (Ping timeout: 265 seconds).
[Sunday 26 September 2010] [20:20:49] <jhawk28> ha, the statically linking only works VS2003 and before based on the comments
[Sunday 26 September 2010] [20:21:29] <jhawk28> and the including the dlls in a sub directory doesn’t work with VS2008-SP1
[Sunday 26 September 2010] [20:23:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [20:43:38] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [20:59:15] Quit bgranger has left this server (Quit: bgranger).
[Sunday 26 September 2010] [21:03:19] Part xrfang has left this channel.
[Sunday 26 September 2010] [22:14:19] Part larrytheliquid has left this channel.
[Sunday 26 September 2010] [22:18:28] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Sunday 26 September 2010] [22:25:28] Quit jhawk28 has left this server (Remote host closed the connection).
[Sunday 26 September 2010] [22:52:02] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [23:12:53] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [23:58:44] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [00:15:22] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [00:21:13] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [00:45:31] Quit mcxx has left this server (Remote host closed the connection).
[Monday 27 September 2010] [01:10:15] Quit pasha_bastov has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [01:11:47] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:21:10] Quit pasha_bastov has left this server (Quit: Leaving).
[Monday 27 September 2010] [01:24:22] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:30:09] Quit guido_g has left this server (Quit: Boom...).
[Monday 27 September 2010] [01:32:49] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:34:01] Quit pasha_bastov has left this server (Ping timeout: 252 seconds).
[Monday 27 September 2010] [01:34:59] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:44:17] Part pieter_hintjens has left this channel.
[Monday 27 September 2010] [01:48:26] Quit zmqircd has left this server (Remote host closed the connection).
[Monday 27 September 2010] [01:52:33] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:53:29] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:53:56] Quit pasha_bastov has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [02:22:49] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [02:39:17] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [03:24:19] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [03:47:37] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [03:56:51] <CIA-20> zeromq2: 03Martin Sustrik 07master * r30c1cba 10/ src/poller_base.cpp : latency optimisation for the case where there are no timers -
http://bit.ly/d8hem9[Monday 27 September 2010] [03:56:51] <CIA-20> zeromq2: 03Martin Sustrik 07master * re2f834d 10/ doc/zmq_getsockopt.txt : ZMQ_FD and ZMQ_EVENTS documentation added -
http://bit.ly/9nWC7q[Monday 27 September 2010] [04:00:08] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [04:32:56] Quit sustrik has left this server (Quit: Leaving).
[Monday 27 September 2010] [04:55:22] Part pasha_bastov has left this channel (“Leaving”).
[Monday 27 September 2010] [04:55:26] Quit plol has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [04:56:52] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [04:57:14] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [05:03:57] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 27 September 2010] [05:17:46] <CIA-20> zeromq2: 03Martin Sustrik 07master * r16c3884 10/ (4 files in 2 dirs): MSVC build fixed -
http://bit.ly/abDASN[Monday 27 September 2010] [05:25:07] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 27 September 2010] [06:05:58] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:19:58] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:26:26] Quit jond has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [06:38:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:40:20] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:42:02] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:47:48] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:50:37] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:52:19] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:55:18] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [07:04:55] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [07:36:09] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [08:11:06] Quit pasha_bastov has left this server (Quit: Leaving).
[Monday 27 September 2010] [08:30:02] Quit plol has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [08:49:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [09:29:25] Quit yawn has left this server (Quit: yawn).
[Monday 27 September 2010] [09:43:47] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [09:44:44] Quit sbahra has left this server (Remote host closed the connection).
[Monday 27 September 2010] [09:47:28] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [09:57:08] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [10:08:45] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [10:28:41] Quit guido_g has left this server (Quit: Boom...).
[Monday 27 September 2010] [10:34:29] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [10:54:21] Quit mytrile has left this server (Quit: Leaving.).
[Monday 27 September 2010] [11:00:08] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [11:12:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:27:38] Quit xla has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [11:28:56] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:29:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 27 September 2010] [11:52:56] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [11:55:02] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:56:03] <shawn42> hello
[Monday 27 September 2010] [11:56:03] Quit zmqircd has left this server (Remote host closed the connection).
[Monday 27 September 2010] [11:57:05] <shawn42> has anyone here done any P2P stuff w/ zeromq?
[Monday 27 September 2010] [12:04:07] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [12:04:07] Nick
drbobbeaty_ is now known as drbobbeaty.
[Monday 27 September 2010] [12:54:26] Quit shawn42 has left this server (Quit: shawn42).
[Monday 27 September 2010] [12:58:31] Quit
antares_ has left this server (Quit:
antares_).
[Monday 27 September 2010] [13:30:28] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [13:57:28] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [14:33:08] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [14:55:58] Quit omarkj has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [15:53:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [16:16:41] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Monday 27 September 2010] [16:24:35] Quit plol has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [16:33:06] Quit
antares_ has left this server (Quit:
antares_).
[Monday 27 September 2010] [16:44:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [16:46:41] Quit ModusPwnens has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [17:02:57] Quit keffo has left this server (Read error: Operation timed out).
[Monday 27 September 2010] [17:05:39] <kristianpaul> Hello
[Monday 27 September 2010] [17:06:48] <kristianpaul> :/
[Monday 27 September 2010] [17:29:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [17:29:39] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 27 September 2010] [17:32:49] Quit zmqircd has left this server (Remote host closed the connection).
*** on Mon Sep 27 17:35:20 2010
[Monday 27 September 2010] [17:35:20] Topic The channel topic is “Welcome!”.
[Monday 27 September 2010] [17:35:24] Mode Channel modes: no messages from outside, topic protection
[Monday 27 September 2010] [17:35:24] Created This channel was created on 2010-02-05 10:44.
[Monday 27 September 2010] [17:37:07] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [17:38:32] Join jond has joined this channel (
~jon@87.74.16.24).
[Monday 27 September 2010] [17:40:13] Quit ModusPwnens has left this server.
[Monday 27 September 2010] [17:50:58] Quit andrewvc has left this server (Ping timeout: 272 seconds).
[Monday 27 September 2010] [18:02:51] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [18:08:21] <ipc> howdy; does anyone know if you can run zmq ‘threadless’; i.e. one process use a TCP-bound zmq socket?
[Monday 27 September 2010] [18:08:53] <ipc> the context docs suggest you shouldn’t init with zero threads unless you’re only using in-process sockets
[Monday 27 September 2010] [18:15:19] Part ipc has left this channel.
[Monday 27 September 2010] [18:33:51] Quit bgranger has left this server (Read error: Operation timed out).
[Monday 27 September 2010] [18:45:19] <kristianpaul> ahh solved
[Monday 27 September 2010] [18:45:24] <kristianpaul> i missed the lib
[Monday 27 September 2010] [18:45:28] <kristianpaul> bye
[Monday 27 September 2010] [18:45:30] Part kristianpaul has left this channel.
[Monday 27 September 2010] [18:57:58] <jhawk28> hello
[Monday 27 September 2010] [19:24:42] Quit aleator has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:24:43] Quit neopallium has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:24:43] Quit Zao has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:53:07] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [20:21:23] Quit andrewvc has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [20:35:53] Quit xla has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [20:51:21] Quit xla has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [20:57:04] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Monday 27 September 2010] [21:16:00] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [21:45:51] Quit jhawk28 has left this server (Remote host closed the connection).
[Monday 27 September 2010] [22:41:48] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [23:39:45] Quit
mw_ has left this server (Quit:
mw_).
[Tuesday 28 September 2010] [00:00:16] Quit grahamalot has left this server (Quit: grahamalot).
[Tuesday 28 September 2010] [01:00:03] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 28 September 2010] [01:15:01] Quit guido_g has left this server (Client Quit).
[Tuesday 28 September 2010] [01:39:30] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rf61921d 10/ src/req.cpp : REQ socket can die when reply is delivered on wrong unerlying connection – fixed -
http://bit.ly/cX2rpW[Tuesday 28 September 2010] [01:46:29] <CIA-20> zeromq2: 03Dhammika Pathirana 07maint * rc1deb22 10/ src/ypipe.hpp : crash when closing an ypipe – fixed -
http://bit.ly/bmxM8U[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Martin Sustrik 07master * rf61921d 10/ src/req.cpp : REQ socket can die when reply is delivered on wrong unerlying connection – fixed -
http://bit.ly/cX2rpW[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Dhammika Pathirana 07master * rc1deb22 10/ src/ypipe.hpp : crash when closing an ypipe – fixed -
http://bit.ly/bmxM8U[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6715f9b 10/ src/ypipe.hpp :
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: Merge branch ‘maint’
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: * maint:
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: crash when closing an ypipe – fixed -
http://bit.ly/cxlL0o[Tuesday 28 September 2010] [01:54:02] Quit plol has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [01:57:13] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [03:18:51] Quit plol has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [03:56:38] Quit neopallium has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [04:03:59] Quit zmqircd has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [04:38:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [05:20:29] <keffo> pieterh, around?
[Tuesday 28 September 2010] [05:21:35] <pieterh> keffo: hi!
[Tuesday 28 September 2010] [05:23:12] <keffo> Busy?
[Tuesday 28 September 2010] [05:24:35] <pieterh> always, but shoot...
[Tuesday 28 September 2010] [05:24:53] <keffo> I’m in the process of solidifying how the network is monitored.. Currently I gather various info & statistics in the loadbalancer which publishes regularly(~5s), and a WPF app subscribing and displaying nice graphs etc
[Tuesday 28 September 2010] [05:25:19] <pieterh> sounds good
[Tuesday 28 September 2010] [05:25:21] <keffo> but I cant really figure out a good way of limiting it.. I dont want to send a complete state every 5s
[Tuesday 28 September 2010] [05:25:22] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 28 September 2010] [05:25:30] <pieterh> how large is the state?
[Tuesday 28 September 2010] [05:26:24] <keffo> Depends, both on what info I decide to publish.. I’d like load, bandwidth usage, connected nodes and their respective stats(cpu/ram/etc), but also an overview of what is happenening, as well as more detailed info for each worker
[Tuesday 28 September 2010] [05:27:07] <pieterh> estimated size? in bytes?
[Tuesday 28 September 2010] [05:27:21] <keffo> geesh, no idea, not in the mb range at least :)
[Tuesday 28 September 2010] [05:27:40] <pieterh> if you have no idea, it’s not sensible to think about optimizing it
[Tuesday 28 September 2010] [05:27:59] <pieterh> so do a back-of-envelope calculation and come up with a figure...
[Tuesday 28 September 2010] [05:28:11] <keffo> I wondered if it was sound to have the monitoring app poll a complete current-state at startup, and then depend on a persistent sub-forwarder to handle deltastates? But that sounds very complicated
[Tuesday 28 September 2010] [05:28:48] <keffo> I can’t really guestimate, the number of nodes can range from the local setup I have here of a few machines, to much larger..
[Tuesday 28 September 2010] [05:28:57] <pieterh> ...
[Tuesday 28 September 2010] [05:29:05] <pieterh> how big is “much larger”?
[Tuesday 28 September 2010] [05:29:18] <keffo> ideally wan :)
[Tuesday 28 September 2010] [05:29:30] <pieterh> please stick a number onto it...
[Tuesday 28 September 2010] [05:29:38] <keffo> but being realistic, perhaps around 20?
[Tuesday 28 September 2010] [05:29:51] <pieterh> and how large would the state be per worker?
[Tuesday 28 September 2010] [05:30:14] <pieterh> please stick a number onto it...
[Tuesday 28 September 2010] [05:30:29] <pieterh> then multiply the two numbers and add something for the overview
[Tuesday 28 September 2010] [05:30:31] <keffo> Basic info(linpack measurements), uptime, average load, around that
[Tuesday 28 September 2010] [05:30:51] <pieterh> come back when you have a total in KB per 5 seconds, a’ight?
[Tuesday 28 September 2010] [05:32:02] <keffo> That’s not what I’m interested in, the assumption here is that no data is published that isn’t “needed”, but I would like to figure out the most efficient means of passing around that data
[Tuesday 28 September 2010] [05:32:29] <keffo> basically how to monitor a distributed system with the least amount of overhead as possible..
[Tuesday 28 September 2010] [05:32:45] <pieterh> this is for research purposes rather than an actual use case...
[Tuesday 28 September 2010] [05:32:53] <keffo> (regardless of what the data actually is)
[Tuesday 28 September 2010] [05:33:03] <keffo> well both I guess, research first, use later :)
[Tuesday 28 September 2010] [05:33:23] <pieterh> well, you can wait until Ch4 of the Guide if you want to
[Tuesday 28 September 2010] [05:33:25] <keffo> Surely this problem has been dealt with before, as loadbalancing has :)
[Tuesday 28 September 2010] [05:33:28] <pieterh> but here is how I’d do it...
[Tuesday 28 September 2010] [05:33:37] <pieterh> - maintain state in the publisher
[Tuesday 28 September 2010] [05:33:47] <pieterh> - apply updates to state and publish updates to pub socket
[Tuesday 28 September 2010] [05:34:01] <pieterh> - in subscriber, request state via req/rep socket
[Tuesday 28 September 2010] [05:34:07] <pieterh> - and also subscribe to updates
[Tuesday 28 September 2010] [05:34:10] <pieterh> - queue incoming updates
[Tuesday 28 September 2010] [05:34:22] <pieterh> - as soon as state arrives, apply updates to state and continue to do this
[Tuesday 28 September 2010] [05:34:22] <keffo> m, that’s what I was thinking as well.
[Tuesday 28 September 2010] [05:35:17] <pieterh> i think it’s robust but need to prove it
[Tuesday 28 September 2010] [05:35:17] <keffo> Things do start to get hairy if the monitor app starts to depend on delta-updates though, like “join/part” of nodes
[Tuesday 28 September 2010] [05:35:56] <keffo> you’re doing this type of stuff for ch 4?
[Tuesday 28 September 2010] [05:35:59] <pieterh> yup
[Tuesday 28 September 2010] [05:36:05] <pieterh> stateful pubsub
[Tuesday 28 September 2010] [05:36:42] <pieterh> or whatever this is properly called...
[Tuesday 28 September 2010] [05:36:55] <keffo> I’ll try it out, see if it behaves well.. :)
[Tuesday 28 September 2010] [05:37:12] <pieterh> feel free to write it up as a recipe or code sample
[Tuesday 28 September 2010] [05:37:18] <pieterh> if i can reuse that for the guide it’ll save me time
[Tuesday 28 September 2010] [05:37:37] <keffo> I’ll let you know for sure yeah
[Tuesday 28 September 2010] [05:38:06] <pieterh> the main reason for this is not so much to save network bandwidth but to allow realtime updates
[Tuesday 28 September 2010] [05:38:21] <keffo> I was aso thinking about ‘history-nodes’ as well.. something appealing about that, someone who keeps track of what’s going on, but isn’t directly part -of- the system
[Tuesday 28 September 2010] [05:38:48] <pieterh> indeed, this work could be totally outsourced to a stateful device
[Tuesday 28 September 2010] [05:38:50] <keffo> non-gonzo network monitoring
[Tuesday 28 September 2010] [05:39:08] <pieterh> with some notion of state + patches
[Tuesday 28 September 2010] [05:39:47] <pieterh> like pair/value updates
[Tuesday 28 September 2010] [05:39:52] <pieterh> hmm, nice
[Tuesday 28 September 2010] [05:39:59] <pieterh> it’s a distributed cache
[Tuesday 28 September 2010] [05:40:08] Quit mytrile has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [05:40:14] <keffo> yes, distributed tuple-storage
[Tuesday 28 September 2010] [05:40:19] <pieterh> yup, that’s the thing
[Tuesday 28 September 2010] [05:40:54] <keffo> I cant help but thinking most source control software face much of the same issues here
[Tuesday 28 September 2010] [05:40:59] <pieterh> in the general solution any node can update its cache
[Tuesday 28 September 2010] [05:41:21] <pieterh> it is a very useful general solution to state distribution
[Tuesday 28 September 2010] [05:41:27] <keffo> oh yeah, one can think of it as cache hits and misses, makes things clearer
[Tuesday 28 September 2010] [05:42:39] <pieterh> i’d start with a simple model, one publisher, many subscribers, pair/value updates
[Tuesday 28 September 2010] [05:43:05] <keffo> It reuses the same mechanisms as the rest
[Tuesday 28 September 2010] [05:44:59] <keffo> maybe I could contribute some C# stuff to go along with ch 4.. I do find it quite lacking on the site..
[Tuesday 28 September 2010] [05:45:56] <pieterh> keffo: we’ve gotten one C# example yesterday, but others would be great
[Tuesday 28 September 2010] [05:46:13] <pieterh> do start at ch1 if you could, it’s quite trivial stuff but useful to newcomers
[Tuesday 28 September 2010] [05:46:34] <keffo> yeah..
[Tuesday 28 September 2010] [05:46:50] <keffo> need a complete binding, for starters :)
[Tuesday 28 September 2010] [05:47:00] <pieterh> the binding is not complete?
[Tuesday 28 September 2010] [05:47:28] <keffo> no, there was no poll for example?
[Tuesday 28 September 2010] [05:47:34] <keffo> or did I somehow get an old version?
[Tuesday 28 September 2010] [05:48:28] <keffo> nope..
[Tuesday 28 September 2010] [05:48:53] <pieterh> who maintains this binding?
[Tuesday 28 September 2010] [05:49:00] <pieterh> ask them to fix it or submit a patch
[Tuesday 28 September 2010] [05:49:07] <keffo> not sure, says sustrik last commit I guess
[Tuesday 28 September 2010] [05:49:32] <pieterh> hmm, the owner of every project should be WRITTEN IN HUGE LETTERS
[Tuesday 28 September 2010] [05:49:41] <pieterh> otherwise it’s kind of dead by definition
[Tuesday 28 September 2010] [05:49:51] <pieterh> nzmq is something layered on top afaics
[Tuesday 28 September 2010] [05:49:58] <pieterh> different API
[Tuesday 28 September 2010] [05:50:05] <keffo> yes, but it does the same lowlevel binding of the dll
[Tuesday 28 September 2010] [05:50:44] <pieterh> a’ight
[Tuesday 28 September 2010] [05:51:10] <keffo> So merging them shouldnt be very difficult
[Tuesday 28 September 2010] [05:51:11] * pieterh is wondering who pays trolls to come talk rubbish whenever there’s a license thread
[Tuesday 28 September 2010] [05:51:22] <keffo> on the list?
[Tuesday 28 September 2010] [05:51:40] <pieterh> “What’s the latest version of ZeroMQ that’s not LGPL? I need to static link in commercial projects and the LGPL is not an option. “
[Tuesday 28 September 2010] [05:53:18] <pieterh> not a single sensible comment on that thread, just trolls coming to complain that [sic] switching to LGPL will kill 0MQ...
[Tuesday 28 September 2010] [05:53:58] <keffo> will it? =)
[Tuesday 28 September 2010] [05:54:18] <pieterh> oh, yes, of course...
[Tuesday 28 September 2010] [05:54:38] <pieterh> that’s why we have to go back in time and switch to the Microsoft Open Software License or whatever...
[Tuesday 28 September 2010] [05:55:22] <keffo> licensing is tricky business, it went and got itself hugely complicated
[Tuesday 28 September 2010] [05:55:31] <keffo> I prefer “mine” and “public domain” :)
[Tuesday 28 September 2010] [05:55:41] <pieterh> it’s not really tricky, just politically sensitive because it involves so much money
[Tuesday 28 September 2010] [05:56:13] <pieterh> every license is the contract on which the community grows
[Tuesday 28 September 2010] [05:56:21] <pieterh> LGPL and GPL are IME proven beyond a reasonable doubt to be the most effective contracts
[Tuesday 28 September 2010] [05:56:27] <pieterh> because they make it impossible to cheat
[Tuesday 28 September 2010] [05:56:34] <pieterh> end.
[Tuesday 28 September 2010] [05:56:51] <PerfDave> Not impossible, see gpl-violations.org. But very difficult ;)
[Tuesday 28 September 2010] [05:57:05] <pieterh> impossible in any sustainable sense
[Tuesday 28 September 2010] [05:57:15] <pieterh> and GPLv3 closed the loopholes people found in GPLv2
[Tuesday 28 September 2010] [05:57:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 28 September 2010] [05:57:15] <keffo> who has the time anyway :)
[Tuesday 28 September 2010] [05:57:23] <pieterh> oh, people love to cheat
[Tuesday 28 September 2010] [05:57:41] <pieterh> but communities die when they get parasited
[Tuesday 28 September 2010] [05:58:09] <pieterh> so these trolls come and complain that LGPL will kill the community when in fact it creates it
[Tuesday 28 September 2010] [05:58:14] <pieterh> sigh.
[Tuesday 28 September 2010] [05:58:20] <keffo> :)
[Tuesday 28 September 2010] [06:03:22] <keffo> what’s the most elaborate zmq based project anyway?
[Tuesday 28 September 2010] [06:07:16] <pieterh> keffo: wow, there are some very elaborate ones out there
[Tuesday 28 September 2010] [06:07:32] <pieterh> but most are so secret that I’d have to kill you after explaining them
[Tuesday 28 September 2010] [06:07:58] <keffo> Oh I just mean scale, not impl. details :)
[Tuesday 28 September 2010] [06:08:12] <keffo> I’d like to know what level I’m at :)
[Tuesday 28 September 2010] [06:08:46] <pieterh> scale: hundreds to thousands of nodes
[Tuesday 28 September 2010] [06:09:03] <pieterh> multiple data centers
[Tuesday 28 September 2010] [06:10:00] <keffo> are they mostly about shuffling data around, or generic compute clusters?
[Tuesday 28 September 2010] [06:10:17] <pieterh> both cases
[Tuesday 28 September 2010] [06:10:47] <keffo> interesting stuff
[Tuesday 28 September 2010] [06:10:56] <pieterh> it’s also growing rapidly
[Tuesday 28 September 2010] [06:11:05] <keffo> I would assume so!
[Tuesday 28 September 2010] [06:11:16] <pieterh> the first 0MQ projects a year or two ago were maybe 10 nodes
[Tuesday 28 September 2010] [06:11:36] <pieterh> i’d say the scale is growing x10 every six months or so
[Tuesday 28 September 2010] [06:12:20] <keffo> It would be nice if the license included algorithmic contributions as opposed to solely sourcecode :)
[Tuesday 28 September 2010] [06:12:46] <pieterh> well, ideas cannot be copyrighted
[Tuesday 28 September 2010] [06:13:04] <keffo> A lot of information was probably gathered during development of those, which ideally should be shared :)
[Tuesday 28 September 2010] [06:13:12] <pieterh> well
[Tuesday 28 September 2010] [06:13:30] <pieterh> whenever possible we do move experience into the open source layers
[Tuesday 28 September 2010] [06:14:07] <pieterh> however there are often valuable business secrets in these algorithms
[Tuesday 28 September 2010] [06:14:23] <pieterh> obviously we do not consider sharing those
[Tuesday 28 September 2010] [06:14:47] <keffo> I was more thinking along the lines of “this common-practice method breaks down under these conditions” etc
[Tuesday 28 September 2010] [06:15:00] <pieterh> this is what the user guide will eventually cover
[Tuesday 28 September 2010] [06:15:36] <pieterh> at least the more common cases
[Tuesday 28 September 2010] [06:15:52] <pieterh> we can also try to document some of the higher level patterns as protocols
[Tuesday 28 September 2010] [06:15:53] <keffo> Some of the things I’ve found annoyingly void so far has been the loadbalancing(which is now covered well enough), and also recursive behaviour, which I think I’ve solved
[Tuesday 28 September 2010] [06:16:10] <pieterh> have you been using the custom routing from Ch3?
[Tuesday 28 September 2010] [06:16:30] <keffo> Yeah, pretty much, but with prioqueues
[Tuesday 28 September 2010] [06:16:37] <pieterh> what is a prioqueue?
[Tuesday 28 September 2010] [06:16:44] <keffo> both for incoming tasks and outgoing results
[Tuesday 28 September 2010] [06:16:50] <keffo> priority queue
[Tuesday 28 September 2010] [06:16:57] <pieterh> ah, so queues in your broker rather than using the socket queues
[Tuesday 28 September 2010] [06:17:03] <keffo> indeed
[Tuesday 28 September 2010] [06:17:17] <pieterh> i just didn’t want to start mucking with data structures
[Tuesday 28 September 2010] [06:17:23] <pieterh> but I think it’s inevitable
[Tuesday 28 September 2010] [06:17:41] <pieterh> i already had to define a zmsg class, will probably define a zqueue class as well
[Tuesday 28 September 2010] [06:17:51] <keffo> messages and tasks have priorities, as well as workers based on a mix of scimark(hardware) and something like a running average network ‘behaviour’
[Tuesday 28 September 2010] [06:17:57] <pieterh> right
[Tuesday 28 September 2010] [06:18:40] <keffo> Another thing I’ve been struggling to figure out a ‘pretty’ solution to is how to make workers present their updates when doing long running jobs..
[Tuesday 28 September 2010] [06:18:58] <pieterh> intermediate updates?
[Tuesday 28 September 2010] [06:18:59] <keffo> That’ sortof ties into what we talked about earlier with the Monitoring app
[Tuesday 28 September 2010] [06:19:00] <keffo> yeah
[Tuesday 28 September 2010] [06:19:06] <pieterh> two threads in the workers, I assume
[Tuesday 28 September 2010] [06:19:17] Quit neopallium has left this server (Read error: Operation timed out).
[Tuesday 28 September 2010] [06:19:17] <pieterh> workers as micro clusters
[Tuesday 28 September 2010] [06:19:31] <pieterh> it’s fractal :-)
[Tuesday 28 September 2010] [06:19:37] <pieterh> every node can be a cluster of nodes
[Tuesday 28 September 2010] [06:19:40] <keffo> leaving it solely up to the designer of the job (ie, they post progress at will) leads to abuse most likely
[Tuesday 28 September 2010] [06:20:07] <pieterh> sounds like you’re solving a lot of interesting problems
[Tuesday 28 September 2010] [06:20:11] <keffo> indeed :)
[Tuesday 28 September 2010] [06:20:21] <keffo> most interesting of all is the fact that a job can post new jobs :)
[Tuesday 28 September 2010] [06:20:22] <pieterh> you should write about it, if you can
[Tuesday 28 September 2010] [06:20:27] <keffo> that’s a brain teaser if anything :)
[Tuesday 28 September 2010] [06:20:28] <keffo> yeah
[Tuesday 28 September 2010] [06:20:38] <keffo> It’s becoming quite large to be honest :)
[Tuesday 28 September 2010] [06:20:46] <pieterh> well, stack-based simulated recursion is an old technique
[Tuesday 28 September 2010] [06:20:50] <keffo> but it’s working well
[Tuesday 28 September 2010] [06:20:56] <pieterh> it’s how we used to do quicksort in cobol
[Tuesday 28 September 2010] [06:21:04] <keffo> stack-based?
[Tuesday 28 September 2010] [06:21:17] <pieterh> your prioqueue can also be a stack
[Tuesday 28 September 2010] [06:21:22] <pieterh> you can push jobs to the front
[Tuesday 28 September 2010] [06:21:28] <pieterh> or to the back
[Tuesday 28 September 2010] [06:21:32] <keffo> ah yeah, that’s the priority of the messages :)
[Tuesday 28 September 2010] [06:21:41] <pieterh> that’s how you simulate recursion
[Tuesday 28 September 2010] [06:21:43] <keffo> he deeper the recursion, the higher the priority
[Tuesday 28 September 2010] [06:21:58] <pieterh> priority is perhaps the wrong metaphor
[Tuesday 28 September 2010] [06:22:12] <pieterh> in fact it’s “push these child jobs” followed by “pop next job and execute”
[Tuesday 28 September 2010] [06:24:09] <pieterh> If you look at section 3, you see how Quicksort (recursion) works using a stack
[Tuesday 28 September 2010] [06:24:33] <keffo> My solution was to have each worker-node (which owns one worker-process per cpu-core) can request to have an additional worker spawned while it sleeps.. So when it posts a child-job, it does so with a higher priority
[Tuesday 28 September 2010] [06:24:50] <keffo> interesting
[Tuesday 28 September 2010] [06:25:25] <pieterh> Leif and I developed these techniques in the 80’s...
[Tuesday 28 September 2010] [06:25:35] <keffo> how old are you? =)
[Tuesday 28 September 2010] [06:25:38] <pieterh> it’s very basic but I think it maps correctly to recursive messaging
[Tuesday 28 September 2010] [06:25:47] <pieterh> not so old
[Tuesday 28 September 2010] [06:25:51] <pieterh> :-)
[Tuesday 28 September 2010] [06:25:58] <pieterh> 47, to be accurate
[Tuesday 28 September 2010] [06:26:17] <keffo> that was grad student times then? =)
[Tuesday 28 September 2010] [06:26:34] <pieterh> nope, first job developing tools for large software houses
[Tuesday 28 September 2010] [06:26:45] <pieterh> event-driven concurrency in cobol
[Tuesday 28 September 2010] [06:27:35] <keffo> obol of all things :)
[Tuesday 28 September 2010] [06:27:57] <keffo> there’s a scary amount of need for cobol developers now
[Tuesday 28 September 2010] [06:28:36] <pieterh> lol...
[Tuesday 28 September 2010] [06:28:54] <pieterh> we used to train people to become cobol developers in like 3 weeks
[Tuesday 28 September 2010] [06:29:13] <keffo> That’s even more scary :)
[Tuesday 28 September 2010] [06:29:22] <keffo> like todays php crowd I guess?
[Tuesday 28 September 2010] [06:29:29] <pieterh> hmm, yeah, I guess
[Tuesday 28 September 2010] [06:29:53] <pieterh> Cobol was good for mediocre programmers, they could make stuff that worked, and didn’t kill the system
[Tuesday 28 September 2010] [06:30:13] <keffo> I’d be very afraid if my bank announced they moved from cobol to php :)
[Tuesday 28 September 2010] [06:30:26] <pieterh> indeed
[Tuesday 28 September 2010] [06:30:48] <keffo> or voting/pacemakers :)
[Tuesday 28 September 2010] [06:44:44] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 28 September 2010] [06:57:18] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [07:01:47] Quit keffo has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [07:28:03] Quit bbigras has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [07:33:31] <CIA-20> jzmq: 03Stefan Majer 07master * rd46166f 10/ src/org/zeromq/ZMQ.java : Merged from upstream -
http://bit.ly/c9pkwW[Tuesday 28 September 2010] [07:33:31] <CIA-20> jzmq: 03Stefan Majer 07master * r05b384d 10/ src/org/zeromq/ZMQ.java : Reduced duplicate Javadoc comments by references to the corresponding setter. -
http://bit.ly/dvaPdr[Tuesday 28 September 2010] [07:33:39] <CIA-20> jzmq: 03Stefan Majer 07master * r1a98406 10/ src/org/zeromq/ZMQ.java : References to the man pages to further clarify the Javadoc. -
http://bit.ly/c5FVwB[Tuesday 28 September 2010] [08:13:38] Join plq has joined this channel (
~plq@78.189.30.218).
[Tuesday 28 September 2010] [08:33:56] Join eintr has joined this channel (
~dln@88.80.182.68).
[Tuesday 28 September 2010] [08:40:18] Quit plq has left this server (Quit: Leaving).
[Tuesday 28 September 2010] [08:49:48] Quit mytrile has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [08:55:33] Quit plol has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [09:13:59] Part plol has left this channel.
[Tuesday 28 September 2010] [09:27:54] <CIA-20> zeromq2: 03Gonzalo Diethelm 07master * r87beaaa 10/ (14 files in 4 dirs): ZMQ_TYPE socket option added -
http://bit.ly/b3HZYe[Tuesday 28 September 2010] [09:30:58] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [09:32:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * r10bb9d0 10/ AUTHORS : Dhammika Pathirana was missing from the AUTOHRS file for some reason – fixed -
http://bit.ly/cz4TZ7[Tuesday 28 September 2010] [09:51:49] Quit sbahra has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [10:07:59] Quit xla has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [10:15:38] Quit jond has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [10:39:50] <CIA-20> zeromq2: 03Steven McCoy 07master * r00cd7d4 10/ (7 files in 3 dirs): Upgrade to OpenPGM-5.0.78 -
http://bit.ly/aKqJjZ[Tuesday 28 September 2010] [10:42:13] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: 03Steven McCopy 07master * r1dc4531 10/ (4 files): (log message trimmed)
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Add assertions to check for OpenPGM calls with invalid parameters.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Assertion to check that pgm_getaddrinfo is actually returning something.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Missing pgm_connect call.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Typo on TOS causing immediate abort.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Placeholder calls for timeouts whilst continuing spin loop functionality.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * OpenPGM v5 now supports reference counting so remove init checks.
[Tuesday 28 September 2010] [11:05:07] Quit zmqircd has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [11:26:22] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 28 September 2010] [11:28:03] <psino> I’m having some issues with understanding how a PUSH socket with no downstream nodes is supposed to behave. There seem to be a difference between the sockets whether they have been used as .bind or .connect
[Tuesday 28 September 2010] [11:28:53] <psino> the connect_socket sends the data (send does not block) even if there are no downstream nodes, but the bind_socket blocks
[Tuesday 28 September 2010] [11:30:48] <psino> is this the expected behaviour? from what I understand after reading
http://api.zeromq.org/zmq_socket.html is that both should have been blocking
[Tuesday 28 September 2010] [11:31:07] <psino> As its said here: “When a ZMQ_PUSH socket enters an exceptional state due to having reached the high water mark for all downstream nodes, or if there are no downstream nodes at all, then any zmq_send(3) operations on the socket shall block until the exceptional state ends or at least one downstream node becomes available for sending; messages are not discarded.”
[Tuesday 28 September 2010] [11:37:27] <sustrik> psino: yes. it works that way
[Tuesday 28 September 2010] [11:38:02] <sustrik> on connect, socket can immediately create a queue to store messages in – even before actual connection is established
[Tuesday 28 September 2010] [11:38:41] <psino> so the manual is incorrect/imprecise?
[Tuesday 28 September 2010] [11:39:10] <sustrik> when binding, the socket has to wait till peers connect, it cannot create a queue itself (it doesn’t even know whether there’ll be a connection in the future)
[Tuesday 28 September 2010] [11:39:43] <sustrik> imprecise, i would say
[Tuesday 28 September 2010] [11:39:55] <sustrik> connect assumed that there’s a peer
[Tuesday 28 September 2010] [11:40:06] <sustrik> exactly 1 peer
[Tuesday 28 September 2010] [11:40:11] <sustrik> bind makes no such assumption
[Tuesday 28 September 2010] [11:41:28] <psino> hmm
[Tuesday 28 September 2010] [11:52:05] <jashmenn> hey can anyone give me some guidance on socket migration between threads
[Tuesday 28 September 2010] [11:52:15] <jashmenn> i’m having a tough time figuring out how that works
[Tuesday 28 September 2010] [11:52:30] <jashmenn> basically i want to close a socket from a thread different than the thread that started it
[Tuesday 28 September 2010] [11:58:14] <mato> sustrik: yo
[Tuesday 28 September 2010] [11:58:48] Quit eintr has left this server (Quit: Leaving.).
[Tuesday 28 September 2010] [11:59:45] <sustrik> mato: hi
[Tuesday 28 September 2010] [11:59:57] <mato> sustrik: i would have appreciated a heads up on the OpenPGM commits
[Tuesday 28 September 2010] [12:00:09] <mato> sustrik: it does touch the build system, of which you declared me the maintainer :-)
[Tuesday 28 September 2010] [12:00:16] <sustrik> ah
[Tuesday 28 September 2010] [12:00:22] <sustrik> i blindly applied the patches :|
[Tuesday 28 September 2010] [12:00:29] <mato> also, isn’t the author of those commits steve mccoy?
[Tuesday 28 September 2010] [12:00:36] <sustrik> yes, he is
[Tuesday 28 September 2010] [12:00:47] <mato> ah, it’s there
[Tuesday 28 September 2010] [12:00:56] <mato> ‘cept you have a typo
[Tuesday 28 September 2010] [12:00:59] <mato> but that’s my fault
[Tuesday 28 September 2010] [12:01:01] <sustrik> ?
[Tuesday 28 September 2010] [12:01:07] * mato is documenting the process
[Tuesday 28 September 2010] [12:01:20] <mato> sustrik: you should not be typing author names manually if at all possible
[Tuesday 28 September 2010] [12:01:32] <sustrik> no idea how to do it automatically
[Tuesday 28 September 2010] [12:01:38] <mato> git-am :-)
[Tuesday 28 September 2010] [12:02:08] <sustrik> good god!
[Tuesday 28 September 2010] [12:02:26] <mato> sustrik: anyway, please do ask for review before committing stuff that touches other people’s designated areas of responsibility
[Tuesday 28 September 2010] [12:02:37] <sustrik> anyway, can you check the part that touches the build system post hoc?
[Tuesday 28 September 2010] [12:02:42] <mato> yes, i can, i will
[Tuesday 28 September 2010] [12:02:47] <sustrik> thanks
[Tuesday 28 September 2010] [12:02:51] <mato> but please don’t do that in future :-)
[Tuesday 28 September 2010] [12:02:54] <mato> including e.g. docs
[Tuesday 28 September 2010] [12:02:56] <sustrik> sure
[Tuesday 28 September 2010] [12:03:03] <sustrik> eek
[Tuesday 28 September 2010] [12:03:07] <mato> eek?
[Tuesday 28 September 2010] [12:03:12] <sustrik> i’ve just committed gonzalos patch
[Tuesday 28 September 2010] [12:03:22] <sustrik> which touchs docs
[Tuesday 28 September 2010] [12:03:36] <sustrik> ZMQ_TYPE socket opt description
[Tuesday 28 September 2010] [12:03:42] <mato> it also has a commit to .gitignore for bin/ for some reason
[Tuesday 28 September 2010] [12:03:51] <mato> as part of gonzalo’s patch
[Tuesday 28 September 2010] [12:03:55] <sustrik> hm
[Tuesday 28 September 2010] [12:04:12] <sustrik> it shouldn’t be there but once it’s there it doesn’t hurt
[Tuesday 28 September 2010] [12:04:15] <mato> sure
[Tuesday 28 September 2010] [12:04:24] <mato> but my point is be careful about what is in a patch
[Tuesday 28 September 2010] [12:04:43] <sustrik> what are we going to do with patches that intersect different areas of functionality?
[Tuesday 28 September 2010] [12:04:53] <sustrik> there’s no clear committer there
[Tuesday 28 September 2010] [12:05:00] <mato> ask the interested parties for review
[Tuesday 28 September 2010] [12:05:03] <mato> that’s how it normally works
[Tuesday 28 September 2010] [12:05:10] <sustrik> ok, will do
[Tuesday 28 September 2010] [12:05:35] <mato> also, if you want my attention quickly a good way to do that is to Cc: me directly as well as sending email to the list
[Tuesday 28 September 2010] [12:05:43] <sustrik> ack
[Tuesday 28 September 2010] [12:05:51] <mato> that way it lands in my INBOX rather than in the auto-filtered-away list folder :-)
[Tuesday 28 September 2010] [12:35:18] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 28 September 2010] [12:42:49] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * re3e6b7f 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java): Added support for getType() and added some missing constants. -
http://bit.ly/at8wUF[Tuesday 28 September 2010] [12:43:04] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [12:57:49] Quit psino has left this server (Quit: psino).
[Tuesday 28 September 2010] [13:02:16] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [13:03:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [13:12:31] Quit psino has left this server (Client Quit).
[Tuesday 28 September 2010] [13:25:37] Quit keffo has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [13:30:21] <vharron> Is there a list of projects using zeromq? I’m trying to understand what it does well by looking at the problems it solves.
[Tuesday 28 September 2010] [13:52:22] <drbobbeaty> vharron: I can’t give you a list of projects, but I can tell you about mine. I’m using ZMQ as a transport layer using OpenPGM and the reliable multicast. I’m using this because what I need is a transport more than a full-fledged messaging system like Tibco or 29West. It’s working out wonderfully in this capacity and we’re moving a lot of messages through the system.
[Tuesday 28 September 2010] [13:53:36] <drbobbeaty> It’s not a real full-blown messaging system - but you can build one with it. I just see it’s real target as a very advanced transport system with a lot of flexibility.
[Tuesday 28 September 2010] [14:04:24] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 28 September 2010] [14:34:33] Quit vharron has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [15:16:22] Quit gavinstark has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [15:51:58] Join jond has joined this channel (
~jon@87.74.16.24).
[Tuesday 28 September 2010] [15:55:18] Quit
mw_ has left this server (Quit:
mw_).
[Tuesday 28 September 2010] [16:20:41] Quit zirpu has left this server (Quit: leaving).
[Tuesday 28 September 2010] [16:23:14] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Tuesday 28 September 2010] [16:29:44] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [16:47:54] <CIA-20> zeromq2: 03Steven McCoy 07master * ra729357 10/ (6 files): more fixes to (e)pgm transport -
http://bit.ly/9E1o2T[Tuesday 28 September 2010] [16:50:08] Quit zmqircd has left this server (Ping timeout: 245 seconds).
*** on Tue Sep 28 16:57:48 2010
[Tuesday 28 September 2010] [16:57:48] Topic The channel topic is “Welcome!”.
[Tuesday 28 September 2010] [16:57:52] Mode Channel modes: no messages from outside, topic protection
[Tuesday 28 September 2010] [16:57:52] Created This channel was created on 2010-02-05 10:44.
[Tuesday 28 September 2010] [17:02:36] Quit zmqircd has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [17:29:16] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 28 September 2010] [17:54:10] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [18:15:07] Quit jashmenn has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [18:28:15] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Tuesday 28 September 2010] [19:14:03] <lluad> If I have multiple clients connecting to a single server over TCP, is there a good idiom for the server to send an unsolicited message to just one of the clients?
[Tuesday 28 September 2010] [19:18:21] Quit rgl has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [19:20:50] <Zao> lluad: When connecting a client, send a connection string to the server if it needs to communicate with you?
[Tuesday 28 September 2010] [19:24:16] Quit lluad has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [19:24:16] Nick
lluad_ is now known as lluad.
[Tuesday 28 September 2010] [19:38:31] Quit lluad has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [19:38:31] Nick
lluad_ is now known as lluad.
[Tuesday 28 September 2010] [20:29:07] Quit rbancroft has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [20:34:15] <novas0x2a> is there an example for how to use zmq_poll properly? This code (where i think i’m using it right?) crashes reliably:
http://codepad.org/uyAlBSOw[Tuesday 28 September 2010] [20:35:28] <novas0x2a> oh wow, i’m lame... ignore that code for a sec..
[Tuesday 28 September 2010] [20:37:54] Quit novas0x2a has left this server (Quit: Leaving.).
[Tuesday 28 September 2010] [21:11:27] Quit iFire has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [21:12:06] Quit miguelito has left this server (Read error: Connection reset by peer).
[Tuesday 28 September 2010] [21:17:59] Quit xla has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [21:24:18] Quit iFire` has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [21:33:26] Quit
RobertLJ_ has left this server (Read error: Operation timed out).
[Tuesday 28 September 2010] [22:28:08] Part neopallium has left this channel (“Leaving”).
[Tuesday 28 September 2010] [22:45:45] Quit lluad has left this server (Quit: lluad).
[Tuesday 28 September 2010] [23:11:38] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Tuesday 28 September 2010] [23:13:49] Quit lluad has left this server (Quit: lluad).
[Wednesday 29 September 2010] [01:11:06] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [01:14:37] Quit iFire has left this server (Remote host closed the connection).
[Wednesday 29 September 2010] [01:32:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 29 September 2010] [01:54:54] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 29 September 2010] [02:11:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 29 September 2010] [02:16:02] Quit rgl has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [02:19:04] Quit jond has left this server (Quit: Leaving).
[Wednesday 29 September 2010] [02:19:38] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [04:03:29] Join eintr has joined this channel (
~dln@88.80.182.68).
[Wednesday 29 September 2010] [04:09:54] <CIA-20> zeromq2: 03Ilja Golshtein 07master * r4482dd7 10/ (AUTHORS include/zmq.hpp): C++ binding destructors don’t throw exceptions -
http://bit.ly/9r4bo6[Wednesday 29 September 2010] [04:20:42] Quit iFire has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [04:59:07] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [05:25:16] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 29 September 2010] [05:39:03] Join omarkj has joined this channel (
~omarkj@81.15.34.139).
[Wednesday 29 September 2010] [05:43:31] Quit keffo has left this server (Ping timeout: 252 seconds).
[Wednesday 29 September 2010] [06:04:01] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [06:17:12] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [06:34:19] Quit iFire` has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [06:39:52] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 29 September 2010] [06:50:58] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [07:34:27] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Wednesday 29 September 2010] [07:35:42] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [08:21:00] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [08:35:11] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [10:00:35] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 29 September 2010] [10:04:42] Quit bgranger has left this server (Ping timeout: 265 seconds).
[Wednesday 29 September 2010] [10:07:06] <llasm> hi !!
[Wednesday 29 September 2010] [10:10:12] <sustrik> llasm: hi
[Wednesday 29 September 2010] [10:49:56] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [10:51:08] <llasm> how to install zmq*
[Wednesday 29 September 2010] [10:51:15] <llasm> i need a tuto
[Wednesday 29 September 2010] [10:53:54] <keffo> define ‘install’?
[Wednesday 29 September 2010] [10:55:48] <pieterh> please tell me if anything is missing or unclear on the instructions there
[Wednesday 29 September 2010] [10:58:50] Quit bgranger has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [11:00:23] Quit eintr has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [11:19:32] Quit bgranger has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [11:21:14] Quit jsimmons has left this server (Ping timeout: 255 seconds).
[Wednesday 29 September 2010] [11:27:49] <llasm> pieterh: i follow this instructions
[Wednesday 29 September 2010] [11:28:20] <llasm> but when try python -c “import zmq”
[Wednesday 29 September 2010] [11:28:40] <llasm> i receive cannot import name _zmq
[Wednesday 29 September 2010] [11:29:29] <pieterh> llasm: ok, you need to also build the Python API
[Wednesday 29 September 2010] [11:30:55] <pieterh> did you install 0MQ into system libraries (‘sudo make install’)?
[Wednesday 29 September 2010] [11:31:23] <pieterh> then you have to specify the system libraries in setup.cfg before doing ‘python setup.py install’
[Wednesday 29 September 2010] [11:31:41] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [11:32:19] <llasm> yes i use this command ‘make install’ with root user
[Wednesday 29 September 2010] [11:33:07] Quit friendshi has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [11:33:11] <pieterh> what does your setup.cfg contain?
[Wednesday 29 September 2010] [11:33:17] <llasm> but i don’t find zeromq2-dev folder in /user/local/
[Wednesday 29 September 2010] [11:34:20] <sustrik> there’s no such folder
[Wednesday 29 September 2010] [11:34:32] <sustrik> you should see /user/local/include/zmq.h for example
[Wednesday 29 September 2010] [11:35:25] <friendshi> llasm: see the section titled ‘Linux’ under ‘Building and installation’ on this page:
http://github.com/zeromq/pyzmq[Wednesday 29 September 2010] [11:37:23] <llasm> my setup.cfg contain
[Wednesday 29 September 2010] [11:37:28] <llasm> library_dirs = /usr/local/zeromq-dev/lib
[Wednesday 29 September 2010] [11:37:32] <llasm> include_dirs = /usr/local/zeromq-dev/include
[Wednesday 29 September 2010] [11:38:58] <friendshi> try this: python setup.py build_ext –rpath=/usr/local/zeromq-dev/lib –inplace
[Wednesday 29 September 2010] [11:39:51] <friendshi> if /usr/local/zeromq-dev/lib is where you actually installed zeromq library
[Wednesday 29 September 2010] [11:40:24] <pieterh> llasm: this is a common mistake, the example setup.cfg is confusing
[Wednesday 29 September 2010] [11:40:45] <pieterh> edit it so that the paths are /usr/local/lib and /usr/local/include
[Wednesday 29 September 2010] [11:42:40] <sustrik> where’s the example?
[Wednesday 29 September 2010] [11:43:22] <pieterh> setup.cfg.template
[Wednesday 29 September 2010] [11:43:30] <sustrik> pyzmq project?
[Wednesday 29 September 2010] [11:43:35] <pieterh> yes
[Wednesday 29 September 2010] [11:43:49] <pieterh> i’m making a quickstarter for Python on the www.zeromq.com download page
[Wednesday 29 September 2010] [11:44:00] <sustrik> ok, let me create a ticket asking to change the example
[Wednesday 29 September 2010] [11:44:09] <pieterh> ack
[Wednesday 29 September 2010] [11:44:16] <llasm> i think that the installation is ok. so i’ve this error when i try : python -c ‘import zmq’
[Wednesday 29 September 2010] [11:44:17] <llasm> ImportError: libzmq.so.0: cannot open shared object file: No such file or directory
[Wednesday 29 September 2010] [11:44:20] <pieterh> also the page should not IMO tell the user to get the zeromq git
[Wednesday 29 September 2010] [11:44:21] <llasm> is normal?
[Wednesday 29 September 2010] [11:44:35] <pieterh> llasm: after zeromq install, ‘sudo ldconfig’
[Wednesday 29 September 2010] [11:45:32] <friendshi> thats the error i was getting before doing the build_ext part from the documentation
[Wednesday 29 September 2010] [11:46:29] <pieterh> friendshi, ah, thanks
[Wednesday 29 September 2010] [11:52:44] <llasm> i excute ‘ldconfig’ but i receive the same error
[Wednesday 29 September 2010] [11:53:31] <sustrik> is there zmq.h in /usr/local/include?
[Wednesday 29 September 2010] [11:55:14] <llasm> yes
[Wednesday 29 September 2010] [11:55:48] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [11:55:52] <pieterh> llasm: did you run ‘sudo python setup.py install’ again?
[Wednesday 29 September 2010] [11:56:04] <llasm> yes
[Wednesday 29 September 2010] [12:00:17] * pieterh has exhausted his pyzmq knowledge
[Wednesday 29 September 2010] [12:00:39] <sustrik> how does you setup.py look like?
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> my setup.cfg contain
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> library_dirs = /usr/local/zeromq-dev/lib
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> include_dirs = /usr/local/zeromq-dev/include
[Wednesday 29 September 2010] [12:04:52] <mato> this would be a problem
[Wednesday 29 September 2010] [12:05:01] <mato> /usr/local/zeromq-dev will not be in a default LD_LIBRARY_PATH
[Wednesday 29 September 2010] [12:05:05] <mato> don’t do that :-)
[Wednesday 29 September 2010] [12:05:27] <mato> if you want things to just work then install everything in /usr/local
[Wednesday 29 September 2010] [12:05:47] <mato> or edit your ld.so.conf / LD_LIBRARY_PATH but normally you don’t want to do that
[Wednesday 29 September 2010] [12:06:41] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [12:12:18] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [12:24:58] <llasm> LD_LIBRARY_PATH is empty. it’s normal?!!
[Wednesday 29 September 2010] [12:25:19] <guido_g> yes
[Wednesday 29 September 2010] [12:26:13] <llasm> ok
[Wednesday 29 September 2010] [12:28:08] <sustrik> have you changed the setup.py?
[Wednesday 29 September 2010] [12:28:55] <sustrik> it should look like this:
[Wednesday 29 September 2010] [12:29:01] <sustrik> library_dirs = /usr/local/lib
[Wednesday 29 September 2010] [12:29:11] <sustrik> include_dirs = /usr/local/include
[Wednesday 29 September 2010] [12:29:47] <ntelford> hi guys
[Wednesday 29 September 2010] [12:30:13] <ntelford> just having a dig around the Java API and I’m wondering if there’s any support for streaming very large messages without having to buffer them in local memory first?
[Wednesday 29 September 2010] [12:30:41] <ntelford> I ask because Socket.send(byte[] msg, long flags) requires the message to be buffered as a byte array first
[Wednesday 29 September 2010] [13:08:01] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [13:24:33] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 29 September 2010] [14:03:08] Quit llasm has left this server (Quit: Quitte).
[Wednesday 29 September 2010] [14:36:38] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Wednesday 29 September 2010] [14:47:16] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 29 September 2010] [15:45:46] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [16:26:43] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [16:43:47] Quit psino has left this server (Quit: psino).
[Wednesday 29 September 2010] [16:53:04] Quit novas0x2a has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [17:31:03] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 29 September 2010] [17:45:31] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [17:45:33] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Wednesday 29 September 2010] [17:47:27] Quit eintr has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [17:48:12] Quit xla has left this server (Ping timeout: 272 seconds).
[Wednesday 29 September 2010] [18:31:43] Quit friendshi has left this server (Remote host closed the connection).
[Wednesday 29 September 2010] [19:34:41] Quit rgl has left this server (Ping timeout: 252 seconds).
[Wednesday 29 September 2010] [19:40:32] <novas0x2a> could someone confirm that I’m using zmq_poll correctly here?
http://codepad.org/iJuspYER I reliably get an uninitialized/corrupted message back out. I’m suspecting a zmq bug, but I’d like to rule out user error.
[Wednesday 29 September 2010] [19:42:22] Quit Tasser has left this server (Read error: Operation timed out).
[Wednesday 29 September 2010] [19:46:23] Quit chmod_work has left this server (Quit: POOF!).
[Wednesday 29 September 2010] [19:55:33] Quit
mw_ has left this server (Ping timeout: 245 seconds).
[Wednesday 29 September 2010] [20:30:18] Join novas0x2a1 has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [20:33:07] Quit novas0x2a has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [20:38:57] <acts_as> Does zeromq have to talk to other zeromq (or
*MQ) hosts? Or can I have zeromq talk to xyz proprietary packets?
[Wednesday 29 September 2010] [20:42:20] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 29 September 2010] [20:47:07] Quit novas0x2a1 has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [20:57:12] <acts_as> anyone here :(
[Wednesday 29 September 2010] [21:01:58] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 29 September 2010] [21:13:59] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 29 September 2010] [21:14:48] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [21:52:07] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Wednesday 29 September 2010] [22:21:10] Quit novas0x2a has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [22:48:37] Quit RobertLJ has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [23:10:58] Quit Tasser has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [23:16:18] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [23:17:43] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [23:37:12] Quit miguelito has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [00:12:15] <jasong_at_apache> I just found out about 0mq
[Thursday 30 September 2010] [00:12:25] <jasong_at_apache> i’m like in love
[Thursday 30 September 2010] [00:19:21] Quit grahamalot has left this server (Quit: grahamalot).
[Thursday 30 September 2010] [00:22:37] Quit lluad has left this server (Quit: lluad).
[Thursday 30 September 2010] [00:30:35] <Pe_Ell> heh, yeah I want to use it but I need a quick working prototype in Perl and the current drivers are blocking...
[Thursday 30 September 2010] [00:30:53] <Pe_Ell> need to look into making an AnyEvent driver but prototype comes first... but I really like the project. :)
[Thursday 30 September 2010] [00:59:22] <lestrrat> Pe_Ell: wait till zeromq 2.1
[Thursday 30 September 2010] [00:59:41] <lestrrat> I’m going to make an AnyEvent adaptor then.
[Thursday 30 September 2010] [01:00:10] <Pe_Ell> yeah, was that you I was talking to about it through email last week?
[Thursday 30 September 2010] [01:00:23] <lestrrat> Ah, duh. must be.
[Thursday 30 September 2010] [01:00:25] <Pe_Ell> I started looking at the C code but I needed to undistract myself... :)
[Thursday 30 September 2010] [01:00:29] <lestrrat> Sounded familiar.
[Thursday 30 September 2010] [01:00:56] <Pe_Ell> 0mq was a shiny object in my time of planning...
[Thursday 30 September 2010] [01:01:07] <lestrrat> :)
[Thursday 30 September 2010] [01:01:11] <Pe_Ell> so doing my initial design testing with AnyEvent::MP
[Thursday 30 September 2010] [01:01:23] <lestrrat> I see.
[Thursday 30 September 2010] [01:01:33] <Pe_Ell> but I want to switch to 0mq after I get it working
[Thursday 30 September 2010] [01:02:34] <lestrrat> I would be willing to build a prototype based on the current master, but I’m also running YAPC::Asia, so I won’t have time to hack on it until it’s over :/
[Thursday 30 September 2010] [01:02:44] <Pe_Ell> np
[Thursday 30 September 2010] [01:03:05] <Pe_Ell> I’ve got a like a week or two of my project before I can get back to it...
[Thursday 30 September 2010] [01:03:15] <lestrrat> so maybe sometime around the end of Oct... unless $dayjob catches me.
[Thursday 30 September 2010] [01:03:16] <lestrrat> righto
[Thursday 30 September 2010] [01:03:21] <Pe_Ell> from what I read look like you can use ZMQ_FD to trigger polling
[Thursday 30 September 2010] [01:03:40] <Pe_Ell> but I didn’t get terribly far into it
[Thursday 30 September 2010] [01:03:47] <lestrrat> right. you just need to pass that FD as a IO object, and register a callback using AE::io
[Thursday 30 September 2010] [01:04:14] <lestrrat> then call ZMQ’s poll function (which I need to change the API a bit...)
[Thursday 30 September 2010] [01:04:41] <Pe_Ell> yeah I was wondering if it would be that simple or not... since they mentioned that it triggers on more than just incoming events
[Thursday 30 September 2010] [01:04:45] <Pe_Ell> so you might not have any events when it triggers
[Thursday 30 September 2010] [01:05:36] <lestrrat> well, there may be caveats I’m missing, but sounds like you should be able to poll() and just have it return immediately w/o any messages.
[Thursday 30 September 2010] [01:05:40] <lestrrat> it wouldn’t matter.
[Thursday 30 September 2010] [01:05:56] <Pe_Ell> that’s what I was hoping
[Thursday 30 September 2010] [01:05:57] <lestrrat> it /would/ matter if it /didn’t/ trigger when there’s a message.
[Thursday 30 September 2010] [01:06:07] <Pe_Ell> yeah
[Thursday 30 September 2010] [01:07:03] <lestrrat> we’ll see. we might need to write a couple of wrappers around the FDs, but as long as we can decorate it like Perl IO object... we should be fine
[Thursday 30 September 2010] [01:08:10] <Pe_Ell> yeah
[Thursday 30 September 2010] [01:35:47] Quit rgl has left this server (Ping timeout: 276 seconds).
[Thursday 30 September 2010] [01:52:19] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [02:11:12] Quit sustrik has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [02:46:20] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: 03Steven McCoy 07master * r96d85b2 10/ (4 files): (log message trimmed)
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Add assertions to check for OpenPGM calls with invalid parameters.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Assertion to check that pgm_getaddrinfo is actually returning something.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Missing pgm_connect call.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Typo on TOS causing immediate abort.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Placeholder calls for timeouts whilst continuing spin loop functionality.
[Thursday 30 September 2010] [03:12:44] <CIA-20> zeromq2: * OpenPGM v5 now supports reference counting so remove init checks.
[Thursday 30 September 2010] [03:12:44] <CIA-20> zeromq2: 03Steven McCoy 07master * rd14be62 10/ (6 files): more fixes to (e)pgm transport -
http://bit.ly/9U1BOZ[Thursday 30 September 2010] [03:12:45] <CIA-20> zeromq2: 03Ilja Golshtein 07master * rec2e859 10/ (AUTHORS include/zmq.hpp): C++ binding destructors don’t throw exceptions -
http://bit.ly/9wp2gs[Thursday 30 September 2010] [03:14:20] Quit guido_g has left this server (Quit: Boom...).
[Thursday 30 September 2010] [03:16:48] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [03:38:04] Join eintr has joined this channel (
~dln@88.80.182.68).
[Thursday 30 September 2010] [04:01:51] Quit jasong_at_apache has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [04:06:19] Quit novas0x2a has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [04:06:43] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Thursday 30 September 2010] [04:21:50] Part jasong_at_apac-1 has left this channel.
[Thursday 30 September 2010] [05:04:03] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [05:13:16] Quit mytrile has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [05:28:20] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 30 September 2010] [05:33:55] <CIA-20> zeromq2: 03Steven McCoy 07master * reaa925e 10/ src/pgm_socket.cpp : FreeBSD doesn’t define MSG_ERRQUEUE – now it’s defined in 0MQ -
http://bit.ly/9p2Kny[Thursday 30 September 2010] [05:35:28] <CIA-20> zeromq2: 03Martin Sustrik 07master * r7881ba1 10/ src/pgm_socket.cpp : minor whitespace fix -
http://bit.ly/dm6tfs[Thursday 30 September 2010] [05:38:03] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [05:59:21] Quit omarkj has left this server (Client Quit).
[Thursday 30 September 2010] [06:12:11] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Thursday 30 September 2010] [06:12:26] <
js__> hi
[Thursday 30 September 2010] [06:13:23] <
js__> im having problem with the importation of the dll into visual studio 2010
[Thursday 30 September 2010] [06:14:15] <
js__> is anybody have face this trouble ?
[Thursday 30 September 2010] [06:14:43] <
js__> for the c# implementation
[Thursday 30 September 2010] [06:16:12] Quit
js__ has left this server (Client Quit).
[Thursday 30 September 2010] [07:32:28] Quit sbahra has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [07:43:00] Part jasong_at_apache has left this channel.
[Thursday 30 September 2010] [07:45:54] Quit unenana has left this server (Client Quit).
[Thursday 30 September 2010] [07:48:58] Part jasong_at_apache has left this channel.
[Thursday 30 September 2010] [08:04:05] <CIA-20> zeromq2: 03Martin Sustrik 07master * r98fa2fa 10/ src/xrep.cpp : fix in XREP’s fair queueing mechanism -
http://bit.ly/8Zh1WS[Thursday 30 September 2010] [08:05:39] Quit pasha_bastov has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [08:05:55] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [08:05:57] Nick
pasha_bastov_ is now known as pasha_bastov.
[Thursday 30 September 2010] [08:06:23] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [08:08:00] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 30 September 2010] [08:08:13] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [08:12:04] Quit pasha_bastov has left this server (Ping timeout: 272 seconds).
[Thursday 30 September 2010] [08:24:48] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 30 September 2010] [08:41:47] <mrshpot> hello, mato
[Thursday 30 September 2010] [08:55:07] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [08:59:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [09:10:58] <CIA-20> zeromq2: 03Max Wolf 07maint * r79aea5f 10/ (AUTHORS src/xrep.cpp): zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/9EVmvj[Thursday 30 September 2010] [09:13:15] <CIA-20> zeromq2: 03Max Wolf 07master * r79aea5f 10/ (AUTHORS src/xrep.cpp): zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/9EVmvj[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: 03Martin Sustrik 07master * rac9b05c 10/ (AUTHORS src/xrep.cpp):
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: Merge branch ‘maint’
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: * maint:
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/dbjJBL[Thursday 30 September 2010] [09:16:23] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: 03Martin Lucina 07master * r8f9080e 10/ src/zmq.cpp :
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: zmq_poll(): Fix busy-loop if timeout is zero
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: Fix a case where zmq_poll() (poll-based version) could go off into a busy-loop
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: if no revents are returned and the timeout passed in is zero. -
http://bit.ly/dq8ZsW[Thursday 30 September 2010] [09:35:58] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [09:53:03] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [09:59:34] <mato> sustrik: are you there?
[Thursday 30 September 2010] [09:59:41] <sustrik> yes
[Thursday 30 September 2010] [09:59:50] <mato> i get an occasional assertion on master
[Thursday 30 September 2010] [09:59:56] <mato> no idea what causes it
[Thursday 30 September 2010] [10:00:02] <sustrik> what’s the assertion?
[Thursday 30 September 2010] [10:00:03] <mato> Assertion failed:
inpipe_ &&
outpipe_ (xrep.cpp:53)
[Thursday 30 September 2010] [10:01:29] <sustrik> that should not happen
[Thursday 30 September 2010] [10:01:32] <mato> will try to get a core dump
[Thursday 30 September 2010] [10:01:33] <sustrik> can we reproduce it?
[Thursday 30 September 2010] [10:01:37] <mato> it’s very random
[Thursday 30 September 2010] [10:01:40] <sustrik> hm
[Thursday 30 September 2010] [10:08:24] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [10:15:47] <mato> sustrik: ok, i have a core file... it comes out of a case with close()
[Thursday 30 September 2010] [10:15:52] <mato> sustrik: so it’s related to socket shutdown
[Thursday 30 September 2010] [10:15:59] <sustrik> ack
[Thursday 30 September 2010] [10:16:08] <sustrik> can i see the backtrace?
[Thursday 30 September 2010] [11:01:41] Quit eintr has left this server (Quit: Leaving.).
[Thursday 30 September 2010] [11:03:00] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Thursday 30 September 2010] [11:39:58] <mikko> ZMQ_TYPE support added to php bindings
[Thursday 30 September 2010] [11:40:08] <mikko> i like the changed semantics with zmq_close
[Thursday 30 September 2010] [11:40:22] <mikko> with languages like php the ‘old’ semantics cause a lot of issues
[Thursday 30 September 2010] [11:40:40] <sustrik> it caused lot of problems to everone
[Thursday 30 September 2010] [11:41:13] <sustrik> but even the new solution is not perfect, zmq_term may now block for prolonged periods of time
[Thursday 30 September 2010] [11:41:32] Quit lluad has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [11:41:49] <sustrik> the only real solution is to have 0mq in kernel space
[Thursday 30 September 2010] [11:41:52] <mato> it still needs discussion and fixing, /me would love to write all the close semantics down properly but is bogged down in work for a client
[Thursday 30 September 2010] [11:42:10] <mato> sustrik: that solution is years away, hence not a solution :)
[Thursday 30 September 2010] [11:42:34] <sustrik> sure
[Thursday 30 September 2010] [11:42:58] <mikko> zmq_close with timeout would be nice i guess
[Thursday 30 September 2010] [11:43:11] <mikko> as intermediate solution
[Thursday 30 September 2010] [11:43:16] <sustrik> yeah, SO_LINGER has to be added
[Thursday 30 September 2010] [11:43:24] <sustrik> it’s on TODO list
[Thursday 30 September 2010] [11:43:31] <mikko> im one step behind
[Thursday 30 September 2010] [11:43:38] <mikko> been on holidays in sunny istanbul
[Thursday 30 September 2010] [11:43:44] <mikko> mainly focusing on kebab and raki
[Thursday 30 September 2010] [11:43:44] <sustrik> lucky you
[Thursday 30 September 2010] [11:44:12] <mikko> how did the meeting go in london?
[Thursday 30 September 2010] [11:44:28] <sustrik> there was a tube strike exactly that day :)
[Thursday 30 September 2010] [11:44:41] <sustrik> the only one who managed to get there was jon dyte
[Thursday 30 September 2010] [11:45:07] <sustrik> pieter was more lucky with chicago meetup
[Thursday 30 September 2010] [11:45:13] <mikko> do you come to london often?
[Thursday 30 September 2010] [11:45:20] <sustrik> depends
[Thursday 30 September 2010] [11:45:35] <sustrik> you’ve been in amsterdam last time, right?
[Thursday 30 September 2010] [11:45:39] <mikko> yeah
[Thursday 30 September 2010] [11:45:52] <sustrik> gonzalo just proposed another meetup in amsterdam
[Thursday 30 September 2010] [11:45:59] <sustrik> won’t you be around by chance?
[Thursday 30 September 2010] [11:46:00] <mikko> yeah, i saw the email
[Thursday 30 September 2010] [11:46:08] <mikko> it looks like im in utrecht on 18th onwards
[Thursday 30 September 2010] [11:46:15] <mikko> which is short way from amsterdam
[Thursday 30 September 2010] [11:46:21] <mikko> s/on/from/
[Thursday 30 September 2010] [11:46:29] <sustrik> nice
[Thursday 30 September 2010] [11:46:34] <mikko> that’s week early though
[Thursday 30 September 2010] [11:46:53] <mikko> it’s either one week or two week job
[Thursday 30 September 2010] [11:47:02] <sustrik> i see
[Thursday 30 September 2010] [11:47:32] <sustrik> well, let’s see whether some other folks are interested in the meetup
[Thursday 30 September 2010] [11:47:59] <mikko> amsterdam is pretty easy to get from most of the europe
[Thursday 30 September 2010] [11:48:05] <mikko> 45 mins flight from here
[Thursday 30 September 2010] [11:48:19] <sustrik> ack
[Thursday 30 September 2010] [11:50:47] <mikko> i would be keen on meet-up if an opportunity comes up
[Thursday 30 September 2010] [11:51:20] <mikko> hmmm, i get a quota of conference days to spend from work
[Thursday 30 September 2010] [11:51:30] <mikko> are you talking in any upcoming event?
[Thursday 30 September 2010] [11:53:12] <sustrik> nope, it’s just a beer event
[Thursday 30 September 2010] [11:54:08] <sustrik> i thought that if 5-6 people would come there it would be worth of spending a day in amsterdam
[Thursday 30 September 2010] [11:54:17] <sustrik> but it’s just social
[Thursday 30 September 2010] [11:55:22] <sustrik> btw, one think we’ve talked about with jon dyte is whether it wouldn’t make sense to organise a small 0mq meetup/conference
[Thursday 30 September 2010] [11:55:58] Quit keffo has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [11:56:09] <mikko> that would be nice
[Thursday 30 September 2010] [11:56:31] <mikko> varnish people do those occasionally
[Thursday 30 September 2010] [11:56:32] <sustrik> it’s purely theoretical so far
[Thursday 30 September 2010] [11:56:40] <mikko> they also brainstorm on features etc
[Thursday 30 September 2010] [11:56:57] <sustrik> ack
[Thursday 30 September 2010] [11:57:05] <sustrik> the logistics is obvious problem
[Thursday 30 September 2010] [11:57:14] <sustrik> people are all around the globe
[Thursday 30 September 2010] [11:57:28] <mikko> that kind of thing
[Thursday 30 September 2010] [11:57:53] <sustrik> yes, something like that
[Thursday 30 September 2010] [11:57:55] <mikko> maybe we could try to organise it around a larger conference
[Thursday 30 September 2010] [11:58:02] <sustrik> one option would be to hold it after fossdem
[Thursday 30 September 2010] [11:58:05] <sustrik> in brussels
[Thursday 30 September 2010] [11:58:11] <sustrik> that’s a really big conference
[Thursday 30 September 2010] [11:58:24] <sustrik> some 2000 attendeed or so
[Thursday 30 September 2010] [11:58:46] <sustrik> held jan/feb iirc
[Thursday 30 September 2010] [11:59:29] <mikko> feb it seems
[Thursday 30 September 2010] [11:59:38] <mikko> they give rooms to distribution miniconferences
[Thursday 30 September 2010] [11:59:42] <mikko> conferences*
[Thursday 30 September 2010] [11:59:54] <mikko> there was no typo actually
[Thursday 30 September 2010] [12:00:05] <mikko> anyway, it seems like a sensible idea
[Thursday 30 September 2010] [12:00:22] <mikko> that way people can attend fossdem and probably get their companies to pay for flights / accomodation
[Thursday 30 September 2010] [12:00:34] <sustrik> there are imatix offices in brussels, so it would be easy
[Thursday 30 September 2010] [12:00:42] <mikko> We hereby invite ideas and proposals for main speakers, and proposals for developer rooms. The call for stands and lightning talks will be done after announcing the accepted devrooms.
[Thursday 30 September 2010] [12:00:54] <mikko> they also provide dev rooms
[Thursday 30 September 2010] [12:01:19] <sustrik> but pieter has a fridge with beer at imatix office :)
[Thursday 30 September 2010] [12:01:37] <mikko> sounds like a plan
[Thursday 30 September 2010] [12:01:45] <mikko> mussels in brussels
[Thursday 30 September 2010] [12:01:49] <sustrik> right
[Thursday 30 September 2010] [12:02:11] <mikko> maybe fire the idea to mailing list and see if people pick up?
[Thursday 30 September 2010] [12:03:00] <omarkj> Oh, a reason to go to Brussels?
[Thursday 30 September 2010] [12:05:23] <mikko> it would be nice to brainstorm and bounce around ideas on how people use zeromq
[Thursday 30 September 2010] [12:07:30] <sustrik> yes, if people are intersted we can arrange that
[Thursday 30 September 2010] [12:13:09] <mikko> seems like pfsense uses the zmq php bindings
[Thursday 30 September 2010] [12:13:39] Quit mrshpot has left this server (Quit: Leaving).
[Thursday 30 September 2010] [12:13:50] <sustrik> what’s that?
[Thursday 30 September 2010] [12:13:55] Quit vagn has left this server (Quit: Ex-Chat).
[Thursday 30 September 2010] [12:13:59] <mikko> it’s a firewall / captive portal
[Thursday 30 September 2010] [12:14:15] <sustrik> interesting
[Thursday 30 September 2010] [12:14:26] <mikko> “pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router.”
[Thursday 30 September 2010] [12:14:42] <mikko> it seems they use zeromq to distribute config changes
[Thursday 30 September 2010] [12:14:42] <sustrik> any idea what do they use it for?
[Thursday 30 September 2010] [12:14:47] <sustrik> i see
[Thursday 30 September 2010] [12:15:15] <sustrik> actually, i would like to see 0MQ devices running on routers one day
[Thursday 30 September 2010] [12:15:17] <mikko> i guess that enables centralised management for firewall clusters
[Thursday 30 September 2010] [12:15:33] <sustrik> but we are not that fat yet
[Thursday 30 September 2010] [12:15:38] <sustrik> far*
[Thursday 30 September 2010] [12:16:40] <sustrik> the idea is that a device sits at the edge of the network (in router) and redistributes messages to the local network
[Thursday 30 September 2010] [12:17:41] <mikko> i think some of the concepts in zeromq are hard to grasp
[Thursday 30 September 2010] [12:17:44] <mikko> for some people
[Thursday 30 September 2010] [12:17:54] <mikko> well, not zeromq specific issue
[Thursday 30 September 2010] [12:18:01] <mikko> more like messaging oriented approach
[Thursday 30 September 2010] [12:18:18] <mikko> and some people seem to insist on ‘persistent’ messaging
[Thursday 30 September 2010] [12:18:44] <sustrik> yes, i am aware of that
[Thursday 30 September 2010] [12:18:51] <sustrik> we need to educate the crowd
[Thursday 30 September 2010] [12:56:19] <jasong_at_apache> one my main dev machine which is OSX (production is ubuntu) i’m getting: ./configure: line 15263: syntax error near unexpected token
`newline’
[Thursday 30 September 2010] [12:56:20] <jasong_at_apache> ./configure: line 15263: ` PKG_CHECK_MODULES(‘
[Thursday 30 September 2010] [12:56:48] <jasong_at_apache> this is optionless ./autogen.sh , ./configure (stock as is)
[Thursday 30 September 2010] [12:58:28] <jasong_at_apache> update: i just found the doc page....
[Thursday 30 September 2010] [12:59:03] <cremes> jasong_at_apache: make sure you have libtool installed on your machine
[Thursday 30 September 2010] [12:59:17] <cremes> if you use macports, do a “port install libtool” and try again
[Thursday 30 September 2010] [12:59:35] <jasong_at_apache> i do, will try! thx cremes
[Thursday 30 September 2010] [13:05:20] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [13:25:05] Quit lluad has left this server (Client Quit).
[Thursday 30 September 2010] [14:00:58] Quit friendshi has left this server (Quit: friendshi).
[Thursday 30 September 2010] [14:16:11] <jasong_at_apache> another issue, now with jzmq
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> libtool: compile: g++ -DHAVE_CONFIG_H -I. -D_REENTRANT -D_THREAD_SAFE -I/Library/Java/Home/include -Wall -g -O2 -MT libjzmq_la-Socket.lo -MD -MP -MF .deps/libjzmq_la-Socket.Tpo -c Socket.cpp -fno-common -DPIC -o .libs/libjzmq_la-Socket.o
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> Socket.cpp: In function ‘jlong Java_org_zeromq_ZMQ_00024Socket_getLongSockopt(JNIEnv*, _jobject*, jint)’:
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> Socket.cpp:103: error: ‘ZMQ_TYPE’ was not declared in this scope
[Thursday 30 September 2010] [14:21:09] <cremes> jasong_at_apache: i don’t think the “official” bindings have been updated to work with 2.1 yet
[Thursday 30 September 2010] [14:21:25] <cremes> if you are just starting to experiment, you might want to stick with 2.0.9
[Thursday 30 September 2010] [14:21:58] <jasong_at_apache> ah
[Thursday 30 September 2010] [14:23:02] <jasong_at_apache> sorry I’m coming across as a total noob, I am monitoring several builds figure i’d just hit you guys up for the obvious
[Thursday 30 September 2010] [14:29:11] Quit omarkj has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [14:30:20] Nick n is now known as Guest28979.
[Thursday 30 September 2010] [14:31:38] <Guest28979> I want to setup a pub-sub system where a client sends a message to a central server that does some authentication and error checking and then publishes that message to all clients.
[Thursday 30 September 2010] [14:32:11] <omarkj> Okay.
[Thursday 30 September 2010] [14:32:24] <Guest28979> I’m doing a request/response pair to send the message to the server, and pub/sub to send out the messages, and it’s working beautifully, except...
[Thursday 30 September 2010] [14:33:00] <Guest28979> These are dynamic hosts, and on ocassion, the server will die and move to a different machine.
[Thursday 30 September 2010] [14:33:42] <Guest28979> Is there an elegant way to do this, or do I need to do non-blocking recv’s with timeouts, and a heartbeat on the PUB/SUB side of things?
[Thursday 30 September 2010] [14:35:54] <Peaceful> Is this a common problem? I’m trying to install zmq and pyzmq for the first time on my OS X 10.6 box, and it looked like it worked, but I get: ImportError: cannot import name _zmq –> when I do “import zmq”
[Thursday 30 September 2010] [14:36:49] <Peaceful> I just downloaded 2.0.9, untarred it, ran ./configure, make, make install ...
[Thursday 30 September 2010] [14:37:05] <Guest28979> Peaceful: How did you install pyzmq?
[Thursday 30 September 2010] [14:37:13] <Guest28979> easy_install?
[Thursday 30 September 2010] [14:37:25] <Peaceful> then downloaded pyzmq (from git), copied setup.cfg (which looked correct), and did ‘sudo python setup.py install’
[Thursday 30 September 2010] [14:37:57] <Guest28979> What was the output of that?
[Thursday 30 September 2010] [14:40:06] * Peaceful is gathering output for pastebin
[Thursday 30 September 2010] [14:45:30] <Peaceful>
http://pastebin.com/8ct0gsif <– and here’s the full output from my ”./configure”, “make”, “sudo make install”
[Thursday 30 September 2010] [14:45:55] <Peaceful> There’s warnings, but it appeared that zmq and pyzmq both successfully installed...but doesn’t work.
[Thursday 30 September 2010] [14:46:44] <guido_g> might be a path issue
[Thursday 30 September 2010] [14:47:10] <Guest28979> I’m hardly a Mac guy, so I’m stabbing in the dark here...
[Thursday 30 September 2010] [14:47:20] <guido_g> dito.
[Thursday 30 September 2010] [14:48:17] <Peaceful> Hmm. There were no mac-specific install instructions for zmq...I checked “INSTALL”. I wonder if zmq itself works okay...
[Thursday 30 September 2010] [14:48:53] <Guest28979> Do you have a directory that looks something like ‘/Library/Python/2.6/site-packages/pyzmq-2.0.9dev-py2.6.egg’?
[Thursday 30 September 2010] [14:49:24] <Peaceful> no, but...
[Thursday 30 September 2010] [14:49:43] <Peaceful> there is this, which is close:
[Thursday 30 September 2010] [14:49:45] <Peaceful> ls /Library/Python/2.6/site-packages/zmq/
[Thursday 30 September 2010] [14:49:49] <Peaceful> __init__.py __init__.pyc _zmq.pxd _zmq.so allocate.pxd buffers.pxd devices.so eventloop log tests
[Thursday 30 September 2010] [14:49:52] <Guest28979> It’s sticking an egg.info file into /Library/Python/2.6/site-package, but where is the egg itself?
[Thursday 30 September 2010] [14:50:05] <Guest28979> Ah, OK.
[Thursday 30 September 2010] [14:50:15] <Guest28979> What’s the mac equivalent of ldconfig?
[Thursday 30 September 2010] [14:50:22] <Peaceful> There’s this: /Library/Python/2.6/site-packages/pyzmq-2.0.9dev-py2.6.egg-info
[Thursday 30 September 2010] [14:50:23] <Guest28979> Try sudo ldconfig
[Thursday 30 September 2010] [14:50:42] <Peaceful> hey!
[Thursday 30 September 2010] [14:50:52] <Peaceful> before I tried that, I thought, maybe I should try a new terminal
[Thursday 30 September 2010] [14:50:58] <Peaceful> opening up a new terminal window worked
[Thursday 30 September 2010] [14:51:02] <Guest28979> Yeah, and the new one has the path.
[Thursday 30 September 2010] [14:51:06] <Peaceful> I didn’t run ldconfig
[Thursday 30 September 2010] [14:51:16] <Guest28979> Or the python path is rebuilt, or what-have-you.
[Thursday 30 September 2010] [14:51:22] <Peaceful> ah, I bet “hash -r” would have fixed it in my original one...
[Thursday 30 September 2010] [14:51:32] <Peaceful> er, nope
[Thursday 30 September 2010] [14:51:39] <Peaceful> okay, I’ll just stick with the new window :)
[Thursday 30 September 2010] [14:51:41] <Guest28979> guido_g was right, it was a path issue.
[Thursday 30 September 2010] [14:51:49] <guido_g> YES!
[Thursday 30 September 2010] [14:51:54] <Guest28979> +1
[Thursday 30 September 2010] [14:51:56] <guido_g> where’S the cookie? :)
[Thursday 30 September 2010] [14:51:58] <Peaceful> guido_g++
[Thursday 30 September 2010] [14:52:05] <Peaceful> ^– cookie
[Thursday 30 September 2010] [14:52:23] * Peaceful hands guido_g his cookie and says “good boy!”
[Thursday 30 September 2010] [14:52:28] <Peaceful> thanks!
[Thursday 30 September 2010] [14:52:32] Nick Guest28979 is now known as nedjl.
[Thursday 30 September 2010] [14:52:46] <nedjl> Let’s see if nickserv lets me have this one.
[Thursday 30 September 2010] [14:52:51] <rgl> how good is zeromq on windows?
[Thursday 30 September 2010] [14:52:52] <Peaceful> Now I’ll go see if I can actually get zmq working. I only discovered its existence last night.
[Thursday 30 September 2010] [14:52:52] * guido_g eats cookie
[Thursday 30 September 2010] [14:53:20] * Peaceful heads off to explore zmq...and eat lunch
[Thursday 30 September 2010] [14:53:22] Quit Peaceful has left this server (Quit: Peaceful).
[Thursday 30 September 2010] [14:53:32] <mikko> rgl: should be fine
[Thursday 30 September 2010] [14:53:33] <nedjl> Peaceful: There’s a “It’s not working” workflow chart that’s very good. Most of my experience with it has involved it just working.
[Thursday 30 September 2010] [14:53:46] <mikko> rgl: as far as i know, cant really vouch for it as i dont use windows
[Thursday 30 September 2010] [14:54:20] <rgl> are there any official windows binaries?
[Thursday 30 September 2010] [14:54:26] * rgl didn’t found them on the homepage
[Thursday 30 September 2010] [14:54:53] <nedjl> I figured out a better way of asking my question – is there a way to get an error when an endpoint goes down? I don’t want zeromq to hide me from this, I need to redo some service discovery to find out where I should connect to instead.
[Thursday 30 September 2010] [14:55:19] <mikko> rgl: you should be able to build easily
[Thursday 30 September 2010] [14:55:38] <rgl> mikko, ok. will do. thx! :)
[Thursday 30 September 2010] [14:55:41] <mikko> rgl: are you looking to use one of the language bindings?
[Thursday 30 September 2010] [14:55:47] <rgl> mikko, C#
[Thursday 30 September 2010] [14:56:19] <guido_g> nedjl: nope, mq is notorously underprovisioned with this kind of stuff
[Thursday 30 September 2010] [14:57:10] <guido_g> it _might_ be that you get a notification through the logging interface in the future, the distant future...
[Thursday 30 September 2010] [14:57:42] <mikko> nedjl: at the moment you would probably implement some sort of heartbeat
[Thursday 30 September 2010] [14:58:21] <mikko> it’s getting busy in here, 66 people
[Thursday 30 September 2010] [14:59:10] <guido_g> complicated by the fact that you don’t know which connection is alive and that there is no way to use a specific underlying connection
[Thursday 30 September 2010] [15:00:25] <nedjl> mikko, guido_g: OK. My current plan to figure out if the sub/pub socket is still alive to send out a heartbeat every few seconds, and kill off all the clients if there is no response.
[Thursday 30 September 2010] [15:00:58] <guido_g> if you have use-cases involving this kind of information, please take the time to mention it on the mailing list
[Thursday 30 September 2010] [15:01:12] <nedjl> The req/res socket I’m polling on the response to a req, and if I don’t get anything after a reasonable amount of time, I redo the service discovery and try again with the new info.
[Thursday 30 September 2010] [15:02:09] Quit omarkj has left this server (Quit: omarkj).
[Thursday 30 September 2010] [15:02:20] <mikko> indeed the hearbeat is problematic
[Thursday 30 September 2010] [15:02:31] <nedjl> I’ve got a server that validates messages indicating events, and then spreads em around to anyone interested. That server will go down from time to time, and I want all the clients to notice and redo the service discovery if appropriate.
[Thursday 30 September 2010] [15:02:38] <mikko> especially in scenarios where for example a specific port is blocked accidentally on a firewall
[Thursday 30 September 2010] [15:02:53] <mikko> as the hearbeat might be going through a different port than main connection
[Thursday 30 September 2010] [15:03:31] <nedjl> These are actually a cluster of EC2 servers, and I’m sending out notifications about changes in the cluster’s state using this mechanism.
[Thursday 30 September 2010] [15:04:37] <mikko> nedjl: and you dont know the peers addresses beforehand?
[Thursday 30 September 2010] [15:04:47] <nedjl> I can’t use multicast, even though it is probably the right solution, because EC2 doesn’t support multicast without some gross tunneling hacks.
[Thursday 30 September 2010] [15:05:06] <mikko> nedjl: doesnt pgm encapsulate the multicast into tcp?
[Thursday 30 September 2010] [15:05:15] <guido_g> no
[Thursday 30 September 2010] [15:05:28] <mikko> even with tcp transport?
[Thursday 30 September 2010] [15:05:34] <nedjl> Machines will get added, machines will get killed. If the machine that gets killed is the one with the server, it’ll get started up somewhere else, and everybody will just do a lookup to find the new location.
[Thursday 30 September 2010] [15:05:36] <guido_g> no
[Thursday 30 September 2010] [15:06:06] <mikko> i might’ve misunderstood pgm in that case
[Thursday 30 September 2010] [15:07:05] <guido_g> pgm is working on ip level, ephm encapsulates it in udp
[Thursday 30 September 2010] [15:07:07] <mikko> time to read rfc 3028
[Thursday 30 September 2010] [15:07:11] <guido_g>
*epgm
[Thursday 30 September 2010] [15:07:18] <mikko> ok
[Thursday 30 September 2010] [15:07:26] <mikko> i thought you could encapsulate inside tcp as well
[Thursday 30 September 2010] [15:07:45] <guido_g> you can’t do real multicast on tcp
[Thursday 30 September 2010] [15:08:18] <nedjl> mikko: 1.1 of RFC 3028 says ” PGM runs over a datagram multicast protocol such as IP multicast [5]. “
[Thursday 30 September 2010] [15:08:34] <nedjl> Curses.
[Thursday 30 September 2010] [15:08:38] <mikko> reading 3028 now
[Thursday 30 September 2010] [15:09:39] <mikko> i guess using tcp wouldnt make much sense indeed
[Thursday 30 September 2010] [15:10:08] <mikko> does epgm work in EC2?
[Thursday 30 September 2010] [15:10:18] <nedjl> So it seems inevitable that I’m looking at lots of NOBLOCK and timeouts.
[Thursday 30 September 2010] [15:10:26] <guido_g> no, because you said multicast doesn’t work
[Thursday 30 September 2010] [15:10:27] <nedjl> Lemme lok.
[Thursday 30 September 2010] [15:10:48] <mikko> guido_g: i don’t know anything about EC2
[Thursday 30 September 2010] [15:10:57] * guido_g knows even less
[Thursday 30 September 2010] [15:11:05] <mikko> but i would imagine there are use-cases for multicast in ‘cloud’
[Thursday 30 September 2010] [15:11:10] <guido_g> <nedjl> I can’t use multicast, even though it is probably the right solution, because EC2 doesn’t support multicast without some gross tunneling hacks.
[Thursday 30 September 2010] [15:11:41] <guido_g> mikko: definitely, but it’s also not easy and might even kill the network
[Thursday 30 September 2010] [15:11:45] <mikko> it might be hard to organize safely
[Thursday 30 September 2010] [15:11:59] <guido_g> so it is wise to disable it
[Thursday 30 September 2010] [15:12:33] <mikko> what’s the current best approach to heartbeats with zmq?
[Thursday 30 September 2010] [15:12:40] <mikko> use a separate oob socket for heartbeat?
[Thursday 30 September 2010] [15:12:50] <nedjl> mikko: That’s probably the question I should be asking.
[Thursday 30 September 2010] [15:12:53] <guido_g> i don’t think that there is a an easy and useful way for real multicast on these sold virtual machines
[Thursday 30 September 2010] [15:13:23] <guido_g> if you use a separate socket your testing this one, not the other one
[Thursday 30 September 2010] [15:13:47] <mikko> guido_g: i would imagine it would also require end to end encryption for the multicast
[Thursday 30 September 2010] [15:14:16] <mikko> i guess it depends on how the cloud is organised, i remember some providers have private clouds with separate VLANs
[Thursday 30 September 2010] [15:14:37] <guido_g> mikko: yes, and every one would be forced to use it... no way
[Thursday 30 September 2010] [15:15:06] <guido_g> won’t work on the “machine on demand” front
[Thursday 30 September 2010] [15:15:07] <mikko> guido_g: not forced of course, but if they cared for the authenticity of the data then yes
[Thursday 30 September 2010] [15:15:29] <mikko> i guess thats almost forcing
[Thursday 30 September 2010] [15:15:56] <guido_g> otoh multicast steals bandwidth from other users, if a subscriber is in their network segment
[Thursday 30 September 2010] [15:16:05] <guido_g> so one has to deal with that too
[Thursday 30 September 2010] [15:16:31] <guido_g> i don’t think real multicast will become widely used in to vapor... oops... cloud world
[Thursday 30 September 2010] [15:16:41] <mikko> and non rate-limited multicast would be a bit scary
[Thursday 30 September 2010] [15:21:40] <jasong_at_apache> anyone here successfully compile jzmq on osx?
[Thursday 30 September 2010] [15:23:18] Quit nedjl has left this server (Quit: Page closed).
[Thursday 30 September 2010] [15:26:50] <mikko> jasong_at_apache: what is the problem you are facing?
[Thursday 30 September 2010] [15:26:55] <mikko> i can give it a spin
[Thursday 30 September 2010] [15:27:18] <jasong_at_apache> on 10.6 i’ve successfully compiled 2.0.9 (download area)
[Thursday 30 September 2010] [15:27:42] <jasong_at_apache> yet with jzmq i get: Socket.cpp: In function ‘jlong Java_org_zeromq_ZMQ_00024Socket_getLongSockopt(JNIEnv*, _jobject*, jint)’:
[Thursday 30 September 2010] [15:27:42] <jasong_at_apache> Socket.cpp:103: error: ‘ZMQ_TYPE’ was not declared in this scope
[Thursday 30 September 2010] [15:27:51] <jasong_at_apache> during make
[Thursday 30 September 2010] [15:27:53] <mikko> ZMQ_TYPE is in 2.1 only
[Thursday 30 September 2010] [15:28:19] <mikko> and i think the type is ‘int’ iirc
[Thursday 30 September 2010] [15:28:20] <jasong_at_apache> i thought i did a checkout -b from a previous commit (i need to do that again)
[Thursday 30 September 2010] [15:28:31] <jasong_at_apache> how far back do i go? there are no tags atm
[Thursday 30 September 2010] [15:28:34] <mikko> the method name suggests that its a long
[Thursday 30 September 2010] [15:28:53] <mikko> jasong_at_apache: 2.1.0 zmq i mean
[Thursday 30 September 2010] [15:28:58] <mikko> jasong_at_apache: the current github trunk
[Thursday 30 September 2010] [15:29:23] <jasong_at_apache> so it’s not worth using the binding with the maint branch?
[Thursday 30 September 2010] [15:29:29] <sustrik> the change to jzmq was committed by gonzalo today
[Thursday 30 September 2010] [15:29:49] <sustrik> so if you grab yesteday’s version it should work with 2.0.9
[Thursday 30 September 2010] [15:30:01] <jasong_at_apache> ah
[Thursday 30 September 2010] [15:30:02] <mikko> jasong_at_apache: if you edit Socket.cpp line 103
[Thursday 30 September 2010] [15:30:13] <jasong_at_apache> and remove it
[Thursday 30 September 2010] [15:30:13] <mikko> #ifdef ZMQ_TYPE around the line should help
[Thursday 30 September 2010] [15:30:23] <jasong_at_apache> and i’ll grab the previous commit
[Thursday 30 September 2010] [15:30:32] <mikko> sustrik: ZMQ_TYPE is ‘int’ ?
[Thursday 30 September 2010] [15:30:32] <sustrik> mikko: yes
[Thursday 30 September 2010] [15:30:51] <mikko> looking at this jzmq uses int64_t
[Thursday 30 September 2010] [15:31:27] <sustrik> gonzalo messed it up
[Thursday 30 September 2010] [15:31:55] <sustrik> please, do report the problem on jzmq bugtracker
[Thursday 30 September 2010] [15:33:53] <jasong_at_apache> docs want straight int
[Thursday 30 September 2010] [15:34:01] <mikko> done
[Thursday 30 September 2010] [15:34:07] <sustrik> thanks
[Thursday 30 September 2010] [15:35:28] * sustrik wonders how to make bindings work with both maint and stable without burdening maintainers with two separate branches
[Thursday 30 September 2010] [15:35:43] <sustrik> maint & master, i meant
[Thursday 30 September 2010] [15:35:53] <jasong_at_apache> maint? who needs maint!
[Thursday 30 September 2010] [15:36:00] <mikko> it’s a lot better now
[Thursday 30 September 2010] [15:36:01] <jasong_at_apache> ;-)
[Thursday 30 September 2010] [15:36:14] <mikko> as there aren’t breakages that often
[Thursday 30 September 2010] [15:36:40] <sustrik> yeah, the versioning contract helped
[Thursday 30 September 2010] [15:36:59] <mikko> are there constants for api version etc at the moment?
[Thursday 30 September 2010] [15:37:17] <sustrik> just a plain 0mq version
[Thursday 30 September 2010] [15:37:39] <sustrik> the api version should translate into library version
[Thursday 30 September 2010] [15:37:45] <sustrik> but that’s not done yet
[Thursday 30 September 2010] [15:37:51] <mikko> i guess it’s easy to create int out of that during configure phase
[Thursday 30 September 2010] [15:38:00] <mikko> something easy to compare in preprocessor macros
[Thursday 30 September 2010] [15:38:25] <mikko> something like #if ZMQ_VERSION_ID > 2001001 #else etc
[Thursday 30 September 2010] [15:38:31] <mikko> 2001001 being 2.1.1
[Thursday 30 September 2010] [15:38:33] <sustrik> mikko: good idea
[Thursday 30 September 2010] [15:41:06] <sustrik> on the other hand, some bindings (C#, lisp) don’t even include the header file
[Thursday 30 September 2010] [15:46:47] <mikko> hmm
[Thursday 30 September 2010] [15:47:02] <mikko> i reckon there still might be value for such a thing
[Thursday 30 September 2010] [15:47:12] <jasong_at_apache> everything compiled correctly (and I’m using trunk all around), also swapped int for int64 on the zmq_type
[Thursday 30 September 2010] [15:47:30] <jasong_at_apache> i meant the other way around there
[Thursday 30 September 2010] [15:49:09] <sustrik> goodo
[Thursday 30 September 2010] [15:49:36] <sustrik> anyway, this kind of thing is a bit scary as people are using the binding is production
[Thursday 30 September 2010] [15:49:40] <sustrik> in*
[Thursday 30 September 2010] [15:49:50] <jasong_at_apache> :-)
[Thursday 30 September 2010] [15:50:04] <sustrik> mikko: yes, it will help for most bindings
[Thursday 30 September 2010] [15:50:23] <sustrik> i’ll discuss it with mato tomorrow, he’s in charge of the build system
[Thursday 30 September 2010] [15:53:37] Quit omarkj has left this server (Quit: omarkj).
[Thursday 30 September 2010] [15:55:13] <novas0x2a> i’m using zmq_poll w/ 2.0.9, REQ/REP pair, tcp or ipc proto, same context (1 process). zmq_poll never seems to notice activity. if i use inproc it works, if i use a spinloop/nonblocking-recv (no poll) it works. should i expect it to work with tcp?
[Thursday 30 September 2010] [15:56:18] <sustrik> yes
[Thursday 30 September 2010] [15:56:46] <sustrik> if you have a probelm, write a simple program showing it and send it to the mailing list
[Thursday 30 September 2010] [15:57:29] <novas0x2a> could you do a quick sanity check to make sure i’m not pebkacing?
http://codepad.org/3Pxo5V1x[Thursday 30 September 2010] [15:58:45] <novas0x2a> (if you want to compile -DUSE_POLL or -DUSE_BLOCK)
[Thursday 30 September 2010] [16:00:31] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * rd215dac 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java):
[Thursday 30 September 2010] [16:00:31] <CIA-20> jzmq: Added support for ZMQ_FD (getFD()) and ZMQ_EVENTS (getEvents()).
[Thursday 30 September 2010] [16:04:16] <sustrik> novas0x2a: the program looks ok after short inspection
[Thursday 30 September 2010] [16:04:57] <sustrik> if you want someone to actually debug it though, you have to strip as much code as possible and send it to the mailing list
[Thursday 30 September 2010] [16:10:51] <jond> novas0x2a: is this the error : raw.c:158: int main(int, char**): Assertion
`ret’ failed.?
[Thursday 30 September 2010] [16:11:39] Quit psino has left this server (Quit: psino).
[Thursday 30 September 2010] [16:11:52] <novas0x2a> jond: that’s the detection of the error, yeah (it should have printed Timeout just before that)
[Thursday 30 September 2010] [16:15:48] <jond> novas0x2a: yep but when you timeout in yr code, you return 0
[Thursday 30 September 2010] [16:15:50] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * r9ccc16b 10/ src/org/zeromq/ZMQ.java : Added support for automatically growing the size of a Poller object. -
http://bit.ly/cCKiIK[Thursday 30 September 2010] [16:17:13] <jond> novas0x2a: and asserting on that doesnt seem to be a great idea
[Thursday 30 September 2010] [16:17:46] <novas0x2a> jond: this is test code, not real code. i’ve been trying to map the conditions under which it doesn’t work, to track it down. clearly, in real code, i would handle a timeout appropriately :)
[Thursday 30 September 2010] [16:19:09] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [16:20:22] Part cremes has left this channel.
[Thursday 30 September 2010] [16:20:33] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Thursday 30 September 2010] [16:20:33] Part cremes has left this channel.
[Thursday 30 September 2010] [16:21:15] <Peaceful> okay, so I made the hello world server-client example, and it worked great until I had to kill the server, and now the server won’t bind to tcp anymore. (???)
[Thursday 30 September 2010] [16:22:12] <Peaceful> ctrl-c wouldn’t stop the server, so I ended up looking up the pid and running “kill $pid”
[Thursday 30 September 2010] [16:22:39] <Peaceful> this has got to be some common newbie problem, I figure...
[Thursday 30 September 2010] [16:23:08] <Peaceful> But usually tcp ports become available again after you’ve killed the original program that was binding the port.
[Thursday 30 September 2010] [16:23:19] <Peaceful> ...in my experience, at least
[Thursday 30 September 2010] [16:25:01] <jasong_at_apache> is there a mailing list keeping track of commits? (does github do this?)
[Thursday 30 September 2010] [16:26:29] <mikko> github does that
[Thursday 30 September 2010] [16:28:02] <jond> novas0x2a: line 63 in yr file. you redefine rc. it should read rc = zmq_poll(item, 1, timeout); and the code works
[Thursday 30 September 2010] [16:30:16] <mikko> sustrik: java handles all options as int64_t (numeric ones)
[Thursday 30 September 2010] [16:30:28] <mikko> so i guess thats not an issue
[Thursday 30 September 2010] [16:31:00] <novas0x2a> jond: oh, man, i knew it was going to be my fault somehow. sigh.
[Thursday 30 September 2010] [16:31:13] <mikko> assuming that sizeof (uint64_t) <= sizeof (int64_t) is a sane assumption
[Thursday 30 September 2010] [16:32:48] <jasong_at_apache> is that only 64bit systems?
[Thursday 30 September 2010] [16:35:43] <mikko> sizeof both is 8 on systems i have access to
[Thursday 30 September 2010] [16:59:49] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Thursday 30 September 2010] [17:17:37] Quit xla has left this server (Ping timeout: 255 seconds).
[Thursday 30 September 2010] [17:28:01] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 30 September 2010] [17:30:12] Quit rossij has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [17:32:02] Quit rgl has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [17:33:18] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Thursday 30 September 2010] [17:57:40] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [18:07:45] Quit keffo has left this server (Ping timeout: 252 seconds).
[Thursday 30 September 2010] [18:09:54] Quit jond has left this server (Quit: Leaving).
[Thursday 30 September 2010] [18:14:20] Quit friendshi has left this server (Quit: friendshi).
[Thursday 30 September 2010] [18:15:55] Quit novas0x2a has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [18:17:05] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [18:18:55] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Thursday 30 September 2010] [18:22:13] Quit plq has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [18:39:35] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 30 September 2010] [19:10:34] * Peaceful doesn’t feel the love today. Oh well, I’m sure it will start working again after a reboot.
[Thursday 30 September 2010] [19:10:44] Quit Peaceful has left this server (Quit: Peaceful).
[Thursday 30 September 2010] [19:22:37] Quit clochette has left this server (Client Quit).
[Thursday 30 September 2010] [19:23:51] Part
js__ has left this channel.
[Thursday 30 September 2010] [19:53:48] Quit _rgl has left this server (Ping timeout: 272 seconds).
[Thursday 30 September 2010] [20:03:18] Quit friendshi has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [20:37:15] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [20:51:34] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [21:07:08] Quit xla has left this server (Ping timeout: 255 seconds).
[Thursday 30 September 2010] [21:42:04] Quit bbigras has left this server (Ping timeout: 252 seconds).
[Thursday 30 September 2010] [22:01:40] Quit ChicagoBud has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [22:18:26] Join ChicagoBud has joined this channel (
~bud@208.78.39.48).
[Thursday 30 September 2010] [22:18:54] Quit jasong_at_apache has left this server (Quit: jasong_at_apache).
[Thursday 30 September 2010] [23:05:45] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [23:10:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [00:12:01] Quit poswald has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [00:48:28] Quit xrfang has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [01:08:16] Quit
xrfang_ has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:36:48] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 01 September 2010] [01:39:43] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:54:38] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:58:11] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47c064f 10/ (src/fq.cpp src/lb.cpp): hangup when closing socket with no pipes attached – fixed -
http://bit.ly/9iy2zX[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * raaa0761 10/ src/pair.cpp : pipe being attached to the PAIR socket during its termination process is immediately asked to terminate itself -
http://bit.ly/bwIpo1[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * rdb73c76 10/ src/pub.cpp : assert when pipe attaches to PUB socket in process of termination – fixed -
http://bit.ly/bCxg7v[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * rce0972d 10/ (src/ctx.cpp src/ctx.hpp src/object.cpp src/object.hpp): context creates an inproc endpoint (‘inproc://log’) to distribute 0MQ’s log messages - http://bit.ly/9YrEbe
[Wednesday 01 September 2010] [02:25:20] Quit sustrik has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [02:29:02] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [02:30:13] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [03:02:01] <guido_g> howdy
[Wednesday 01 September 2010] [03:05:49] <guido_g> cool, logging...
[Wednesday 01 September 2010] [03:09:41] <guido_g> does the core itself use the logging?
[Wednesday 01 September 2010] [03:10:06] * guido_g looks at the patch
[Wednesday 01 September 2010] [03:20:18] <guido_g> hmmm
[Wednesday 01 September 2010] [03:20:23] <guido_g> doesn’t seem so
[Wednesday 01 September 2010] [03:26:11] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [03:27:08] <sustrik> only the core can use the logging
[Wednesday 01 September 2010] [03:27:27] <sustrik> however, atm it’s not logging anything
[Wednesday 01 September 2010] [03:27:46] <sustrik> logs are to be added gradually
[Wednesday 01 September 2010] [03:31:32] <guido_g> well... better then nothing :)
[Wednesday 01 September 2010] [04:50:11] <Mikael_H_Kjaer> Sustrik: I don’t have a small example that triggers the bug I’ve posted about on the mailing list.
[Wednesday 01 September 2010] [04:51:04] <sustrik> hi mikael
[Wednesday 01 September 2010] [04:51:10] <sustrik> that’s a problem
[Wednesday 01 September 2010] [04:51:30] <sustrik> from inspecting the backtrace it looks like it’s not a 0mq problem
[Wednesday 01 September 2010] [04:51:37] <Mikael_H_Kjaer> I know. I have been trying to boil it down
[Wednesday 01 September 2010] [04:51:49] <sustrik> maybe a memory overwrite?
[Wednesday 01 September 2010] [04:52:05] <sustrik> some random code overwriting the socket?
[Wednesday 01 September 2010] [04:52:07] <Mikael_H_Kjaer> Maybe. Currently I am not hitting the issue anymore
[Wednesday 01 September 2010] [04:52:55] <Mikael_H_Kjaer> But I shouldn’t have any wild pointers or memory accesses in my code. I am currently using smart pointers for everything and my cycle checker seems to think I am okay too.
[Wednesday 01 September 2010] [04:54:07] <Mikael_H_Kjaer> I haven’t valgrinded my code in a few weeks but I will soon try it with zmq added in.
[Wednesday 01 September 2010] [04:54:47] <sustrik> ok. good
[Wednesday 01 September 2010] [04:55:24] <sustrik> btw, maybe try to compile with debug into so that backtrace is more informative
[Wednesday 01 September 2010] [04:56:37] <Mikael_H_Kjaer> I do :) what additional information do you want
[Wednesday 01 September 2010] [04:58:46] <sustrik> hm
[Wednesday 01 September 2010] [04:59:14] <sustrik> can you possibly add a printf statement to socket’s destructor
[Wednesday 01 September 2010] [04:59:36] <sustrik> so that we know whether the socket was destructed or overwritten?
[Wednesday 01 September 2010] [04:59:58] <sustrik> printing out this pointer would be good so that we can correlate it with the backtrace
[Wednesday 01 September 2010] [05:00:05] <Mikael_H_Kjaer> Not a problem
[Wednesday 01 September 2010] [05:00:22] <Mikael_H_Kjaer> I will get back with more. Any other printf’s you want?
[Wednesday 01 September 2010] [05:01:53] <sustrik> not yet
[Wednesday 01 September 2010] [05:02:02] <sustrik> let’s first see whrther the socket is destructed
[Wednesday 01 September 2010] [05:06:57] Quit
miguelito_ has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [05:12:38] <Mikael_H_Kjaer> sustrik: I can’t get the error to show up any more. I believe my fix of waiting for all sends to finish before attempting a shutdown of zmq has resolved it.
[Wednesday 01 September 2010] [05:12:58] <sustrik> hm
[Wednesday 01 September 2010] [05:13:29] <sustrik> ok, so let’s leave it for now
[Wednesday 01 September 2010] [05:13:40] <sustrik> if you ever see it again, we’ll continue
[Wednesday 01 September 2010] [05:16:20] <Mikael_H_Kjaer> Agreed, I was thinking the same.
[Wednesday 01 September 2010] [05:23:43] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 01 September 2010] [05:28:05] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 01 September 2010] [05:41:42] Quit Mikael_H_Kjaer has left this server (Quit: Page closed).
[Wednesday 01 September 2010] [05:52:02] <mcxx> however, in this setup, the broker is the single point of failure
[Wednesday 01 September 2010] [05:52:54] <mcxx> can the architecture be altered somehowe? e.g. can the clients connect to multiple brokers so that when one goes down, they switch to a working one?
[Wednesday 01 September 2010] [05:54:50] <sustrik> yes, you can load balance via multiple brokers
[Wednesday 01 September 2010] [05:55:08] <sustrik> just start 2 brokers
[Wednesday 01 September 2010] [05:55:17] <sustrik> and connect all components to both of them
[Wednesday 01 September 2010] [05:55:35] <sustrik> you also have to set HWM on clients
[Wednesday 01 September 2010] [05:55:52] <sustrik> so that messages are not queued for the unavailable broker indefinitely
[Wednesday 01 September 2010] [05:56:46] Quit guido_g has left this server (Client Quit).
[Wednesday 01 September 2010] [05:57:14] <mcxx> what’s HWM? Google doesn’t show reasonable results
[Wednesday 01 September 2010] [05:57:44] <mcxx> oh, I see noe
[Wednesday 01 September 2010] [05:57:45] <mcxx> now
[Wednesday 01 September 2010] [05:59:24] Quit guido_g has left this server (Disconnected by services).
[Wednesday 01 September 2010] [05:59:30] Nick OldNugget is now known as guido_g.
[Wednesday 01 September 2010] [06:39:04] <mcxx> when I set HWM on the client and one of the brokers goes down, will the clients’ sockets find out the broker is no longer available and automatically use others?
[Wednesday 01 September 2010] [06:39:45] <guido_g> huh? what broker?
[Wednesday 01 September 2010] [06:40:12] <guido_g> you mean the pipes of the underlying connections?
[Wednesday 01 September 2010] [06:40:32] <mcxx> guido_g: it’s a follow up to my previous question
[Wednesday 01 September 2010] [06:40:38] <guido_g> as far as i know, full pipes (hwm reached) are skipped
[Wednesday 01 September 2010] [06:41:08] <guido_g> i see
[Wednesday 01 September 2010] [06:41:38] <guido_g> still, full pipes are skipped
[Wednesday 01 September 2010] [06:43:58] <mcxx> re-reading that helps :)
[Wednesday 01 September 2010] [06:46:03] <guido_g> :)
[Wednesday 01 September 2010] [07:13:31] Quit rbraley has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [07:26:08] Quit sustrik has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [07:32:01] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6a0c323 10/ (builds/msvc/libzmq/libzmq.vcproj src/ctx.cpp): MSVC build fixed -
http://bit.ly/cEMTGs[Wednesday 01 September 2010] [07:39:18] <pieterh> I’ve posted a blog that hopefully clears up the naming confusion for once and for all...
[Wednesday 01 September 2010] [07:39:27] <pieterh> anyone from Denmark here?
[Wednesday 01 September 2010] [07:40:01] Quit ivan has left this server (Quit: Coyote finally caught me).
[Wednesday 01 September 2010] [07:43:28] <guido_g> :)
[Wednesday 01 September 2010] [08:04:50] <pieterh> sustrik: random question about 2.1
[Wednesday 01 September 2010] [08:05:21] <sustrik> yes?
[Wednesday 01 September 2010] [08:07:08] <pieterh> how solid is it?
[Wednesday 01 September 2010] [08:07:18] <pieterh> should I start to be using it for the examples in the guide?
[Wednesday 01 September 2010] [08:08:01] <sustrik> it haven’t been tested extensively
[Wednesday 01 September 2010] [08:08:06] <sustrik> but the API is the same
[Wednesday 01 September 2010] [08:08:34] <sustrik> so, for the guide, it’s irrelevant which one you use
[Wednesday 01 September 2010] [08:08:46] <pieterh> there’s the ‘insert random sleep here’ which i’d like to remove
[Wednesday 01 September 2010] [08:08:55] <sustrik> ah, yes, that can go with 2.1
[Wednesday 01 September 2010] [08:09:00] <pieterh> i’ll start to test it then
[Wednesday 01 September 2010] [08:09:11] <pieterh> it’d be fun to release a 2.1.0 unstable IMO
[Wednesday 01 September 2010] [08:09:12] <sustrik> btw, there’s a test suite in the package now
[Wednesday 01 September 2010] [08:09:24] <pieterh> especially if you have stuff like sys:// in there...
[Wednesday 01 September 2010] [08:09:24] <sustrik> so if you write a test
[Wednesday 01 September 2010] [08:09:28] <sustrik> jsut put it in there
[Wednesday 01 September 2010] [08:10:05] <sustrik> well, the infrastructure is there, but no notifications are passed atm
[Wednesday 01 September 2010] [08:10:08] <pieterh> :-) i saw some emails on that but missed the original discussion
[Wednesday 01 September 2010] [08:10:15] <pieterh> great idea to have a test suite!
[Wednesday 01 September 2010] [08:10:28] <sustrik> it’s in tests subdir
[Wednesday 01 September 2010] [08:10:34] <sustrik> just put your file there
[Wednesday 01 September 2010] [08:10:39] <sustrik> add it to Makefila.am
[Wednesday 01 September 2010] [08:10:45] <sustrik> and make check will run it
[Wednesday 01 September 2010] [08:10:47] <pieterh> ack
[Wednesday 01 September 2010] [08:11:42] <guido_g> ah, while talking about tests
[Wednesday 01 September 2010] [08:12:10] <guido_g> i plan to write the basic tests for the missing socket types over the weekend
[Wednesday 01 September 2010] [08:12:30] <sustrik> missing socket types?
[Wednesday 01 September 2010] [08:12:40] <sustrik> ah, those not yet in test suite
[Wednesday 01 September 2010] [08:12:47] <guido_g> afaik only req/rep and pair are tested atm
[Wednesday 01 September 2010] [08:12:48] <sustrik> nice
[Wednesday 01 September 2010] [08:13:04] <pieterh>
ZMQ_[Wednesday 01 September 2010] [08:13:05] <guido_g> if there are any corner cases that should be tested, please leave me a note
[Wednesday 01 September 2010] [08:13:14] <pieterh> the secret socket typ
[Wednesday 01 September 2010] [08:13:18] <pieterh>
*type
[Wednesday 01 September 2010] [08:13:27] <pieterh> if you can’t spell it, you can’t use it...
[Wednesday 01 September 2010] [08:13:35] <guido_g> pieterh seems to be in a funny mood today
[Wednesday 01 September 2010] [08:13:36] <guido_g> :)
[Wednesday 01 September 2010] [08:13:56] * pieterh had too many beers with mato last night, his head is not switched on right
[Wednesday 01 September 2010] [08:13:56] <sustrik> you really got the letter or have you made it up?
[Wednesday 01 September 2010] [08:14:05] <pieterh> that’s a real letter
[Wednesday 01 September 2010] [08:14:19] <pieterh> i couldn’t invent something _that_ ... unique
[Wednesday 01 September 2010] [08:14:22] <guido_g> justifies the number of beers you had
[Wednesday 01 September 2010] [08:14:40] <pieterh> that’s why the guide made that strange reference to annoying the Danes
[Wednesday 01 September 2010] [08:15:07] <guido_g> ahhh
[Wednesday 01 September 2010] [08:15:54] <pieterh> next we’re going to get email from the mathematicians accusing us of soiling Bourbaki’s name
[Wednesday 01 September 2010] [08:16:42] <pieterh> guido_g: what language are you using for the tests?
[Wednesday 01 September 2010] [08:16:52] <guido_g> c++
[Wednesday 01 September 2010] [08:17:09] <guido_g> but they can be in any language the core understands
[Wednesday 01 September 2010] [08:17:11] <pieterh> there’s one example that would work for push/pull and pair
[Wednesday 01 September 2010] [08:17:35] <guido_g> for these sockets we have tests already
[Wednesday 01 September 2010] [08:17:38] <pieterh> the mtrelay
[Wednesday 01 September 2010] [08:17:58] <pieterh> you just said only req/rep and pair...?
[Wednesday 01 September 2010] [08:18:21] <guido_g> oh sorry
[Wednesday 01 September 2010] [08:18:33] <guido_g> parsed the pub/sub wrong
[Wednesday 01 September 2010] [08:18:46] <guido_g> push/pull
[Wednesday 01 September 2010] [08:18:48] <pieterh> agh, I realized that example in fact doesn’t use push/pull finally, i switched to using pair
[Wednesday 01 September 2010] [08:18:51] * guido_g needs new glases
[Wednesday 01 September 2010] [08:18:58] * pieterh needs a new head
[Wednesday 01 September 2010] [08:19:26] <guido_g> and i thought they do have good beer in .sk
[Wednesday 01 September 2010] [08:19:43] <pieterh> it’s ok, not real beer like we make in Belgium
[Wednesday 01 September 2010] [08:19:54] <guido_g> ah
[Wednesday 01 September 2010] [08:20:02] <pieterh> kind of halfway between real beer (belgian) and fake beer (anglo)
[Wednesday 01 September 2010] [08:20:08] <pieterh> just kidding
[Wednesday 01 September 2010] [08:20:19] <pieterh> it’s the home of beer, of course
[Wednesday 01 September 2010] [08:20:57] <pieterh> ZeroMQ is a big hit in Japan it seems
[Wednesday 01 September 2010] [08:21:15] <pieterh> RT !heavenshell: perl ZeroMQ
[Wednesday 01 September 2010] [08:21:27] <guido_g> ahh... sure
[Wednesday 01 September 2010] [08:21:42] <sustrik> there are some japanese guys working on the perl binding afaiu
[Wednesday 01 September 2010] [08:21:43] <pieterh> I think they’re complaining about the spelling... :-/
[Wednesday 01 September 2010] [08:21:51] <pieterh> my Japanese isn’t so good anymore
[Wednesday 01 September 2010] [08:22:01] Quit jond has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [08:22:13] <sustrik> btw “where 0mq goes, google cannot follow” is a nice slogan
[Wednesday 01 September 2010] [08:22:16] * pieterh wonders about MQ...
[Wednesday 01 September 2010] [08:22:28] <sustrik> wrt distributed vs. centralised services
[Wednesday 01 September 2010] [08:22:38] <sustrik> privacy etc.
[Wednesday 01 September 2010] [08:22:55] <sustrik> MQ would be read NoMQ :)
[Wednesday 01 September 2010] [08:25:48] <pieterh> Hehe, “can’t follow this”...
[Wednesday 01 September 2010] [08:27:34] <pieterh> we need to prepare an ultrasecret joke for next April 1st
[Wednesday 01 September 2010] [08:27:40] <pieterh> where we rename 0MQ to MQ
[Wednesday 01 September 2010] [08:27:54] <pieterh> get lots of people to blog and twitter about it
[Wednesday 01 September 2010] [08:27:59] <pieterh> confuse the heck out of everyone...
[Wednesday 01 September 2010] [08:31:16] <pieterh> sustrik: could you send me a .zip of the zeromq2 master (clean checkout)?
[Wednesday 01 September 2010] [08:31:29] <pieterh> i’m on a slow 2G connection and have failed 4 times to do a git pull
[Wednesday 01 September 2010] [08:31:43] <pieterh> maybe email will work better
[Wednesday 01 September 2010] [08:36:32] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Wednesday 01 September 2010] [08:36:32] Notice -NickServ- You are now identified for travlr.
[Wednesday 01 September 2010] [08:36:32] CTCP Received Version request from frigg.
[Wednesday 01 September 2010] [08:36:33] Topic The channel topic is “Welcome!”.
[Wednesday 01 September 2010] [08:36:40] Mode Channel modes: no messages from outside, topic protection
[Wednesday 01 September 2010] [08:36:40] Created This channel was created on 2010-02-05 10:44.
[Wednesday 01 September 2010] [08:39:25] <pieterh> sustrik: recv’d! thanks... git seems to have a problem working across slow links
[Wednesday 01 September 2010] [08:39:44] <pieterh> too much chatter or something... they aren’t using 0MQ....
[Wednesday 01 September 2010] [09:02:00] Quit poswald has left this server (Quit: poswald).
[Wednesday 01 September 2010] [09:10:29] <pieterh> sustrik: what’s your view of when 2.1.0 would be releasable?
[Wednesday 01 September 2010] [09:10:46] <pieterh> would it help if I stress tested it with some real applications?
[Wednesday 01 September 2010] [09:10:58] <pieterh> btw, it all build and passed the test suite fine, of course
[Wednesday 01 September 2010] [09:11:02] <pieterh>
*built
[Wednesday 01 September 2010] [09:11:48] <mikko> you can run tests in language bindings as well
[Wednesday 01 September 2010] [09:11:49] <guido_g> not a real suprise, the tests are really basic
[Wednesday 01 September 2010] [09:11:58] <mikko> they probably contain fair amount of tests
[Wednesday 01 September 2010] [09:12:58] <pieterh> gudio_g: i’d like to try breakage tests...
[Wednesday 01 September 2010] [09:13:13] <guido_g> nevertheless do we need good test coverage in the core
[Wednesday 01 September 2010] [09:13:27] <pieterh> yes, we can’t refer up the stack
[Wednesday 01 September 2010] [09:13:42] <pieterh> core can’t test using language bindings, this is up to the binding author/community
[Wednesday 01 September 2010] [09:13:55] <guido_g> pieterh: what does that mean?
[Wednesday 01 September 2010] [09:14:02] <guido_g> breakage tests
[Wednesday 01 September 2010] [09:14:17] <pieterh> well, stuff like send message, close, check that it’s delivered or not according to socket policy
[Wednesday 01 September 2010] [09:14:29] <pieterh> exceed HWM, check messages are delayed/discarded
[Wednesday 01 September 2010] [09:14:36] <pieterh> send invalid stuff, check 0MQ does not die
[Wednesday 01 September 2010] [09:14:50] <pieterh> do bad operations, invalid socket mixes, illegal sequences
[Wednesday 01 September 2010] [09:15:01] <pieterh> all the stuff normal apps do all the time... :-)
[Wednesday 01 September 2010] [09:15:23] <guido_g> ahh ok
[Wednesday 01 September 2010] [09:15:33] <guido_g> normal tests :)
[Wednesday 01 September 2010] [09:15:42] <pieterh> normal regression is of course also very useful
[Wednesday 01 September 2010] [09:15:45] <pieterh> “it still works”
[Wednesday 01 September 2010] [09:15:57] <pieterh> but “i can’t break it” is also valuable
[Wednesday 01 September 2010] [09:16:02] <guido_g> that you get with every build and make check
[Wednesday 01 September 2010] [09:16:18] <guido_g> the regression test
[Wednesday 01 September 2010] [09:16:25] <pieterh> yes
[Wednesday 01 September 2010] [09:16:37] <guido_g> ok, would be nice to have a list of tests to do
[Wednesday 01 September 2010] [09:16:49] <pieterh> it’s sometimes possible to collect test cases for issues
[Wednesday 01 September 2010] [09:16:51] <guido_g> i’ll go over it as time permits and write the test programs
[Wednesday 01 September 2010] [09:17:07] <pieterh> oh, sorry, i’m not trying to propose work... :-/
[Wednesday 01 September 2010] [09:17:20] <guido_g> it has to be done
[Wednesday 01 September 2010] [09:17:25] <pieterh> i’m more than happy to help though it would be in C not C++
[Wednesday 01 September 2010] [09:17:47] <pieterh> well, let’s start by collecting specs?
[Wednesday 01 September 2010] [09:17:53] <pieterh> sorry s/start/continue/
[Wednesday 01 September 2010] [09:18:04] <guido_g> yes
[Wednesday 01 September 2010] [09:18:19] <pieterh> perhaps this can help language binding authors make coherent test sets too
[Wednesday 01 September 2010] [09:18:29] <guido_g> to make it leight and easy to start with, a simple collection of ideas what to test would be ok i think
[Wednesday 01 September 2010] [09:18:30] <pieterh> i.e. standardized test set spec
[Wednesday 01 September 2010] [09:18:36] <pieterh> yes
[Wednesday 01 September 2010] [09:19:06] <pieterh> it would give users a tool for validating bindings
[Wednesday 01 September 2010] [09:19:16] <pieterh> “has test suite”
[Wednesday 01 September 2010] [09:20:12] <pieterh> actually it should then be written both in C++ and C
[Wednesday 01 September 2010] [09:20:25] <pieterh> i volunteer to make the C test suite then
[Wednesday 01 September 2010] [09:20:31] * pieterh opens his big mouth again
[Wednesday 01 September 2010] [09:22:25] <guido_g> :)
[Wednesday 01 September 2010] [09:22:38] <guido_g> i’d like to stick with c++ for now
[Wednesday 01 September 2010] [09:22:50] <guido_g> it’s hard enough to relearn it
[Wednesday 01 September 2010] [09:23:25] <pieterh> sure, so we will double up the test suite, that also gives people examples to compare?
[Wednesday 01 September 2010] [09:23:37] <pieterh> sustrik: do you have any problem with a C test suite in parallel to the C++ one?
[Wednesday 01 September 2010] [09:23:50] <pieterh> if it’s a stupid idea, i’m happy to retract it... :-)
[Wednesday 01 September 2010] [09:26:12] <guido_g> hehe
[Wednesday 01 September 2010] [09:26:24] * guido_g is leaving the office
[Wednesday 01 September 2010] [09:26:36] <guido_g> see you later
[Wednesday 01 September 2010] [09:27:19] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 01 September 2010] [09:29:46] <CIA-20> zeromq2: 03Martin Sustrik 07master * r651c1ad 10/ (src/ctx.cpp src/socket_base.cpp): sys transport introdced; inproc://log moved to sys://log - http://bit.ly/bmO6tM
[Wednesday 01 September 2010] [09:29:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0fe7d3c 10/ (builds/msvc/libzmq/libzmq.vcproj src/ctx.cpp): conflicts resolved -
http://bit.ly/aV1tLE[Wednesday 01 September 2010] [09:49:28] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [10:02:12] <guido_g> re
[Wednesday 01 September 2010] [10:03:13] <sustrik> pieterh: C test suite is OK
[Wednesday 01 September 2010] [10:03:31] <sustrik> but afaik the files have to have .cpp extension
[Wednesday 01 September 2010] [10:03:44] <sustrik> so that autotools know it should link c++ runtime with it
[Wednesday 01 September 2010] [10:04:11] <sustrik> (needed by 0mq library, in case of static linking)
[Wednesday 01 September 2010] [10:04:15] <pieterh> yes, indeed
[Wednesday 01 September 2010] [10:04:24] <pieterh> great
[Wednesday 01 September 2010] [10:04:42] <guido_g> why is the logging now a special transport?
[Wednesday 01 September 2010] [10:04:47] <pieterh> will have to use a naming convention to avoid conflicts, NP
[Wednesday 01 September 2010] [10:05:05] <sustrik> guido_g: two reasons
[Wednesday 01 September 2010] [10:05:39] <sustrik> 1. it’s a different beast, communicating with 0mq internals rather than communicating with a socket in a different thread
[Wednesday 01 September 2010] [10:05:50] <sustrik> 2. full backward compatibility
[Wednesday 01 September 2010] [10:06:14] <guido_g> ok
[Wednesday 01 September 2010] [10:06:49] <pieterh> sustrik: i see one problem with making it a separate transport
[Wednesday 01 September 2010] [10:06:54] <guido_g> how will users attach to this type of transport?
[Wednesday 01 September 2010] [10:07:08] <sustrik> guido_g: connect
[Wednesday 01 September 2010] [10:07:10] <guido_g> just a SUB w/ sys://log?
[Wednesday 01 September 2010] [10:07:16] <sustrik> pieterh: what problem?
[Wednesday 01 September 2010] [10:07:25] <sustrik> guido_g: yes
[Wednesday 01 September 2010] [10:07:31] <pieterh> take the syslog service as an analogy
[Wednesday 01 September 2010] [10:07:31] <guido_g> great! thanks
[Wednesday 01 September 2010] [10:07:40] <pieterh> it’s a bus, any component can participate
[Wednesday 01 September 2010] [10:08:08] <pieterh> ok, that’s not quite accurate but I mean you might want to allow arbitrary components to inject traffic into this
[Wednesday 01 September 2010] [10:08:19] <sustrik> i don’t :)
[Wednesday 01 September 2010] [10:08:24] <pieterh> and have it picked up by log clients without them needing to be changed
[Wednesday 01 September 2010] [10:08:25] <guido_g> :)
[Wednesday 01 September 2010] [10:08:46] <sustrik> what i mean is how would you scale that kind of thing?
[Wednesday 01 September 2010] [10:08:52] <sustrik> right now it’s easy
[Wednesday 01 September 2010] [10:09:06] <pieterh> it’s inproc or ipc, what’s the scaling issue?
[Wednesday 01 September 2010] [10:09:06] <sustrik> it’s constrained to the process
[Wednesday 01 September 2010] [10:09:17] <sustrik> the business logic
[Wednesday 01 September 2010] [10:09:24] <pieterh> attach device to sub, export as pub... np
[Wednesday 01 September 2010] [10:09:35] <sustrik> what you want with a business logic is a distributed logging service
[Wednesday 01 September 2010] [10:09:44] <guido_g> ack
[Wednesday 01 September 2010] [10:09:58] <guido_g> the way with a forwarder is ok
[Wednesday 01 September 2010] [10:10:07] <pieterh> i can play devil’s advocate here
[Wednesday 01 September 2010] [10:10:22] <sustrik> pieterh: i have no idea how you would implement it
[Wednesday 01 September 2010] [10:10:28] <pieterh> it looks like you are claiming a semantic difference in order to solve a naming clash
[Wednesday 01 September 2010] [10:10:40] <pieterh> it’s unclear what the value is of a new transport
[Wednesday 01 September 2010] [10:10:55] <pieterh> no clear problem you’re solving wrt to using inproc:
[Wednesday 01 September 2010] [10:10:55] <sustrik> you mean between sys & inproc or between 0mq logging and application logging?
[Wednesday 01 September 2010] [10:10:58] <pieterh> or ipc:
[Wednesday 01 September 2010] [10:12:22] <pieterh> sustrik: ... sorry, I don’t follow this thread
[Wednesday 01 September 2010] [10:12:36] <sustrik> pieterh> it looks like you are claiming a semantic difference in order to solve a naming clash
[Wednesday 01 September 2010] [10:12:45] <pieterh> ah, yes
[Wednesday 01 September 2010] [10:12:55] <pieterh> between sys: and inproc or ipc
[Wednesday 01 September 2010] [10:13:15] <pieterh> is sys: like inproc or like ipc?
[Wednesday 01 September 2010] [10:13:28] <sustrik> it’s different from both
[Wednesday 01 September 2010] [10:13:38] <sustrik> it has no peer actually
[Wednesday 01 September 2010] [10:13:55] <pieterh> yes, but can you connect to it from another process?
[Wednesday 01 September 2010] [10:13:59] <pieterh> is it an in or ex-process endpoint?
[Wednesday 01 September 2010] [10:14:12] <sustrik> it’s process-scoped
[Wednesday 01 September 2010] [10:14:19] <pieterh> so it’s like inproc
[Wednesday 01 September 2010] [10:14:30] <sustrik> sure, same scope
[Wednesday 01 September 2010] [10:14:33] <sustrik> different semantics
[Wednesday 01 September 2010] [10:15:01] <pieterh> now if i am being evil and minimalist i’ll ask what the value is in defining a totally new protocol instead of a protected namespace within inproc
[Wednesday 01 September 2010] [10:15:15] <pieterh> given that every new construct is something expensive to document, understand...
[Wednesday 01 September 2010] [10:15:45] <sustrik> same question: why not make ipc a separate namespace within tcp?
[Wednesday 01 September 2010] [10:15:51] <pieterh> but the semantics fit perfectly to an inproc SUB socket from the users’ POV
[Wednesday 01 September 2010] [10:16:09] <sustrik> no: there’s no peer socket
[Wednesday 01 September 2010] [10:16:23] <pieterh> that’s an implementation detail
[Wednesday 01 September 2010] [10:16:29] <pieterh> that’s behind the wall, and not relevant to users
[Wednesday 01 September 2010] [10:16:35] <pieterh> it looks like there is a peer socket
[Wednesday 01 September 2010] [10:16:38] <sustrik> ok, let me explain the internal mechanism of sys
[Wednesday 01 September 2010] [10:16:48] <pieterh> ah, hang on
[Wednesday 01 September 2010] [10:16:55] <pieterh> i’m at the API level
[Wednesday 01 September 2010] [10:17:04] <pieterh> internal mechanisms should not be relevant
[Wednesday 01 September 2010] [10:17:09] <guido_g> this is what sys: says imho, don’t expect the usual, this comming from the kernel
[Wednesday 01 September 2010] [10:17:24] <pieterh> but why?
[Wednesday 01 September 2010] [10:17:27] <pieterh> why the surprise?
[Wednesday 01 September 2010] [10:17:47] <guido_g> i can’t see any, sorry
[Wednesday 01 September 2010] [10:17:55] <pieterh> why is the kernel a ‘special’ component with its own transport that looks exactly like inproc: but has a different name?
[Wednesday 01 September 2010] [10:17:57] <pieterh> it’s surprising
[Wednesday 01 September 2010] [10:18:03] <sustrik> it’s administration/business logic distinction
[Wednesday 01 September 2010] [10:18:10] <sustrik> inproc carries business logic
[Wednesday 01 September 2010] [10:18:17] <sustrik> sys carries admin data
[Wednesday 01 September 2010] [10:18:18] <pieterh> not necessarily
[Wednesday 01 September 2010] [10:18:24] <sustrik> necessarily
[Wednesday 01 September 2010] [10:18:24] <pieterh> as soon as i make a framework over 0MQ
[Wednesday 01 September 2010] [10:18:30] <pieterh> then inproc carries admin data
[Wednesday 01 September 2010] [10:18:39] <guido_g> for an application using the core
[Wednesday 01 September 2010] [10:18:50] <guido_g> not the mq core itself
[Wednesday 01 September 2010] [10:18:58] <pieterh> ok... perhaps it would help if there was some significant semantic difference between admin data and business data
[Wednesday 01 September 2010] [10:19:01] <pieterh> such as priority
[Wednesday 01 September 2010] [10:19:02] <guido_g> i see sys: as the core namespace
[Wednesday 01 September 2010] [10:19:07] <pieterh> or blocking/ error handling
[Wednesday 01 September 2010] [10:19:19] <pieterh> guido_g: as a namespace this is a lousy solution IMO
[Wednesday 01 September 2010] [10:19:36] <pieterh> namespaces should be orthogonal to transports
[Wednesday 01 September 2010] [10:19:36] <guido_g> pieterh: why?
[Wednesday 01 September 2010] [10:19:57] <pieterh> well, this makes it impossible to run a separate logger to inspect a running process
[Wednesday 01 September 2010] [10:20:13] <pieterh> every single application needs a special thread that exports the sys: data over ipc: or tcp:
[Wednesday 01 September 2010] [10:20:17] <sustrik> the main problem i have with the whole log thing is that people are now going to use it for their business logic and thus break the patterns
[Wednesday 01 September 2010] [10:20:26] <sustrik> if so, i’ll rather remove it completely
[Wednesday 01 September 2010] [10:20:28] <pieterh> sustrik: no, you can prevent that easily
[Wednesday 01 September 2010] [10:20:36] <sustrik> ?
[Wednesday 01 September 2010] [10:20:36] <guido_g> sustrik: hold it, we need it!
[Wednesday 01 September 2010] [10:20:38] <pieterh> you can prevent that trivially
[Wednesday 01 September 2010] [10:20:43] <pieterh> use a protected name space
[Wednesday 01 September 2010] [10:20:56] <guido_g> like sys: ? :)
[Wednesday 01 September 2010] [10:21:00] <pieterh> recall in amqp how exchanges beginning with amq. were protected
[Wednesday 01 September 2010] [10:21:10] <pieterh> transports are not namespaces... :-(
[Wednesday 01 September 2010] [10:21:20] <guido_g> sure
[Wednesday 01 September 2010] [10:21:27] <sustrik> what i mean is people are now going to ask me to report disconnections via sys
[Wednesday 01 September 2010] [10:21:29] <pieterh> inproc://zmq/log and ipc://zmq/log would be perfect
[Wednesday 01 September 2010] [10:21:31] <pieterh> allows me to start separate logger
[Wednesday 01 September 2010] [10:21:38] <sustrik> so that they can handle each connection separately
[Wednesday 01 September 2010] [10:21:59] <pieterh> perhaps
[Wednesday 01 September 2010] [10:22:08] <sustrik> to prevent that, the admin part should be as much separated from the business part as possible
[Wednesday 01 September 2010] [10:22:09] <pieterh> that’s a use case but it’s possibly not the right solution
[Wednesday 01 September 2010] [10:22:18] <pieterh> hmm
[Wednesday 01 September 2010] [10:22:22] <sustrik> different API would be ideal
[Wednesday 01 September 2010] [10:22:36] <guido_g> but the application developer doe not know what makes sys: work technically, so the obvious similarity to inproc might be wrong
[Wednesday 01 September 2010] [10:22:36] <sustrik> but that’s too much work
[Wednesday 01 September 2010] [10:22:38] <pieterh> sustrik: do you see the value in allowing this namespace to work over both ipc and inproc?
[Wednesday 01 September 2010] [10:22:48] <sustrik> no
[Wednesday 01 September 2010] [10:23:07] <sustrik> it’s just adminstration stuff
[Wednesday 01 September 2010] [10:23:37] <pieterh> i don’t think logging should be mixed with out of band control of sockets
[Wednesday 01 September 2010] [10:23:45] <pieterh> that’s a different issue
[Wednesday 01 September 2010] [10:23:47] <sustrik> right
[Wednesday 01 September 2010] [10:23:55] <sustrik> but it’s an obvious trap
[Wednesday 01 September 2010] [10:24:11] <pieterh> avoidable trap
[Wednesday 01 September 2010] [10:24:13] <sustrik> i think someone have already suggested something like that
[Wednesday 01 September 2010] [10:24:19] <pieterh> yes
[Wednesday 01 September 2010] [10:24:27] <sustrik> i can avoid it by saying “NO”
[Wednesday 01 September 2010] [10:24:36] <sustrik> as i am core maintainer atm
[Wednesday 01 September 2010] [10:24:46] <pieterh> indeed
[Wednesday 01 September 2010] [10:24:53] <pieterh> but that’s not going away, your veto
[Wednesday 01 September 2010] [10:24:57] <sustrik> no mixing of business logic and administration
[Wednesday 01 September 2010] [10:25:04] <pieterh> anyone can fork 0MQ and add whatever they like
[Wednesday 01 September 2010] [10:25:10] <pieterh> but it won’t be the official version
[Wednesday 01 September 2010] [10:25:16] <sustrik> sure, that’s ok
[Wednesday 01 September 2010] [10:25:19] <pieterh> unless and until you vote it in
[Wednesday 01 September 2010] [10:25:27] <pieterh> no exceptions to this, period.
[Wednesday 01 September 2010] [10:25:58] <guido_g> are we still at the logging thing or do i have to move myself to a safe place?
[Wednesday 01 September 2010] [10:26:00] <sustrik> good, so unless i hear a good argument for mixing business logic and maitenence/debugging stuff, i would keep it apart
[Wednesday 01 September 2010] [10:26:18] <sustrik> it’s logging still :)
[Wednesday 01 September 2010] [10:26:27] <pieterh> sustrik: i’d like, really love, to be able to do logging from a separate process
[Wednesday 01 September 2010] [10:26:42] <pieterh> otherwise every application becomes N x more complex
[Wednesday 01 September 2010] [10:26:42] <sustrik> write an app that does that
[Wednesday 01 September 2010] [10:26:43] <sustrik> np
[Wednesday 01 September 2010] [10:26:50] <sustrik> 5 lines of code
[Wednesday 01 September 2010] [10:27:14] <pieterh> you said it was in process
[Wednesday 01 September 2010] [10:27:25] <pieterh> did I misunderstand you?
[Wednesday 01 September 2010] [10:27:35] <sustrik> you can create a device that would read from sys and republish on tcp
[Wednesday 01 September 2010] [10:27:44] <sustrik> you can even use forwarder
[Wednesday 01 September 2010] [10:28:19] <sustrik> when you do that you are effectively converting 0mq admin data into business logic data
[Wednesday 01 September 2010] [10:28:20] <sustrik> which is ok
[Wednesday 01 September 2010] [10:28:30] <sustrik> but at 0mq level i need them separated
[Wednesday 01 September 2010] [10:28:36] <sustrik> to avoid contamination
[Wednesday 01 September 2010] [10:28:50] <guido_g> if anyone can access the internal data flow w/o authorization it would be a nightmare
[Wednesday 01 September 2010] [10:29:05] <sustrik> you mean the logs?
[Wednesday 01 September 2010] [10:29:09] <guido_g> yes
[Wednesday 01 September 2010] [10:29:16] <sustrik> :(
[Wednesday 01 September 2010] [10:29:26] <sustrik> no authorisation as for now
[Wednesday 01 September 2010] [10:29:33] <sustrik> otoh
[Wednesday 01 September 2010] [10:29:38] <sustrik> it’s in-process only
[Wednesday 01 September 2010] [10:29:49] <sustrik> so nobody from outside can read from sys
[Wednesday 01 September 2010] [10:29:52] <guido_g> given the “one forwarder per program” model, the program decides on what to publish and where etc.
[Wednesday 01 September 2010] [10:30:31] <guido_g> sustrik: sure, but as far as i understood, pieterh wanted access from an external process w/o a forwarder inbetween
[Wednesday 01 September 2010] [10:30:47] <sustrik> yes, that was the original discussion
[Wednesday 01 September 2010] [10:31:10] <sustrik> actually it would be almost impossible from implementation point of view
[Wednesday 01 September 2010] [10:31:20] <sustrik> that’s black magic going on in the background :)
[Wednesday 01 September 2010] [10:31:21] <guido_g> details! :)
[Wednesday 01 September 2010] [10:31:44] <sustrik> well, sys is technically different from all the other sockets
[Wednesday 01 September 2010] [10:31:56] <sustrik> maybe pgm excluded
[Wednesday 01 September 2010] [10:32:02] <sustrik> becasue it’s a real bus
[Wednesday 01 September 2010] [10:32:08] <sustrik> ie. multiple writers
[Wednesday 01 September 2010] [10:32:13] <sustrik> multiple readers
[Wednesday 01 September 2010] [10:32:19] <sustrik> no device in the middle
[Wednesday 01 September 2010] [10:32:28] <guido_g> as i said “<guido_g> but the application developer doe not know what makes sys: work technically, so the obvious similarity to inproc might be wrong”
[Wednesday 01 September 2010] [10:32:56] <guido_g> therefore the sys: transport makes sense in my eyes
[Wednesday 01 September 2010] [10:33:09] <sustrik> yes, that was my argument as well
[Wednesday 01 September 2010] [10:33:14] <guido_g> good
[Wednesday 01 September 2010] [10:33:33] <guido_g> because i really like the logging
[Wednesday 01 September 2010] [10:33:48] <sustrik> it’s good but can be dangerous if misused
[Wednesday 01 September 2010] [10:34:04] <guido_g> mq is notoriously weak on giving information on what’s going on
[Wednesday 01 September 2010] [10:34:14] <sustrik> :|
[Wednesday 01 September 2010] [10:34:23] <guido_g> that’s a thing nearly every library has
[Wednesday 01 September 2010] [10:34:35] <sustrik> right
[Wednesday 01 September 2010] [10:34:52] <sustrik> that’s because it’s much more async that probably any other library around
[Wednesday 01 September 2010] [10:35:02] <sustrik> zillions of moving parts
[Wednesday 01 September 2010] [10:35:09] <sustrik> even without proper synchronisation
[Wednesday 01 September 2010] [10:35:12] <guido_g> and for real distributed applications there is rule 0: log everything, all the time
[Wednesday 01 September 2010] [10:35:21] <sustrik> right
[Wednesday 01 September 2010] [10:35:27] <sustrik> so, this should solve it
[Wednesday 01 September 2010] [10:35:32] <sustrik> hopefully
[Wednesday 01 September 2010] [10:35:47] <guido_g> it’s at least a clean interface
[Wednesday 01 September 2010] [10:36:01] <pieterh> sustrik: makes sense but it’s a shame that we’d need a dedicated thread in every app to export log data
[Wednesday 01 September 2010] [10:36:10] <pieterh> still, this can be extended over time
[Wednesday 01 September 2010] [10:36:23] <pieterh> minimalism demands I shut up now :-)
[Wednesday 01 September 2010] [10:36:41] <pieterh> though a new question comes to mind...
[Wednesday 01 September 2010] [10:36:56] <pieterh> if a bus transport is useful for 0MQ internally
[Wednesday 01 September 2010] [10:37:01] <pieterh> surely it is also useful for applications as well
[Wednesday 01 September 2010] [10:37:07] <pieterh> :-)
[Wednesday 01 September 2010] [10:37:09] <sustrik> it doesn’t scale
[Wednesday 01 September 2010] [10:37:17] <sustrik> it can work on level on 1 process
[Wednesday 01 September 2010] [10:37:24] <sustrik> even on LAN
[Wednesday 01 September 2010] [10:37:35] <sustrik> once you try to stretch it further it breaks
[Wednesday 01 September 2010] [10:38:00] <sustrik> the number of transfers grows quadratically with increasding number of components
[Wednesday 01 September 2010] [10:38:38] <sustrik> pieterh: ad dedicated thread
[Wednesday 01 September 2010] [10:38:42] <pieterh> sustrik: as an inproc bus, I meant
[Wednesday 01 September 2010] [10:38:45] <sustrik> yes, that’s why it’s a bus
[Wednesday 01 September 2010] [10:38:46] <pieterh> bus://
[Wednesday 01 September 2010] [10:38:58] <sustrik> to avoid speacial thread
[Wednesday 01 September 2010] [10:39:14] <sustrik> yes, can be
[Wednesday 01 September 2010] [10:39:37] <sustrik> but i like that sys is explicit abour “don’t pass your business logic through this”
[Wednesday 01 September 2010] [10:39:48] <pieterh> for some reason ‘magic system stuff you use here’ seems not ideal
[Wednesday 01 September 2010] [10:40:16] <pieterh> it’s like the core gets to use magic stuff the apps don’t get
[Wednesday 01 September 2010] [10:40:28] <sustrik> yes
[Wednesday 01 September 2010] [10:40:31] <guido_g> that’s why it’s the core
[Wednesday 01 September 2010] [10:40:37] <sustrik> because core doesn’t have to scale
[Wednesday 01 September 2010] [10:40:42] <sustrik> applications do
[Wednesday 01 September 2010] [10:41:36] <pieterh> imo you will get people demanding access to this transport for their essential internal signalling
[Wednesday 01 September 2010] [10:41:51] <pieterh> but i have no proof, just baseless speculation
[Wednesday 01 September 2010] [10:42:07] <sustrik> i expect it will be so
[Wednesday 01 September 2010] [10:42:19] <sustrik> thus i want it be named something really scary
[Wednesday 01 September 2010] [10:42:30] <guido_g> but the line is drawn “sorry, no access for you here”
[Wednesday 01 September 2010] [10:42:34] <pieterh> e.g. “how do I do a clean shutdown”... answer: signal every thread
[Wednesday 01 September 2010] [10:42:44] <guido_g> as with system calls into the kernel
[Wednesday 01 September 2010] [10:42:45] <pieterh> bah. that was the idea with PAIR as well
[Wednesday 01 September 2010] [10:42:49] <sustrik> that’s baked into 0mq as well
[Wednesday 01 September 2010] [10:42:52] <sustrik> ETERM
[Wednesday 01 September 2010] [10:42:55] <pieterh> but it turns out we really need PAIR for some use cases
[Wednesday 01 September 2010] [10:43:15] <sustrik> ok, the use case for pair:
[Wednesday 01 September 2010] [10:43:18] <pieterh> yeah, i know, it’s a bad example
[Wednesday 01 September 2010] [10:43:26] <sustrik> you have an application that does accounting and HR
[Wednesday 01 September 2010] [10:43:34] <sustrik> you want to split it into two applications
[Wednesday 01 September 2010] [10:43:37] <sustrik> one does accounting
[Wednesday 01 September 2010] [10:43:40] <sustrik> other does HR
[Wednesday 01 September 2010] [10:43:52] <sustrik> they have a PAIR socket between them
[Wednesday 01 September 2010] [10:44:10] <sustrik> because each of the peer is singleton
[Wednesday 01 September 2010] [10:44:16] <sustrik> and doesn’t scale
[Wednesday 01 September 2010] [10:44:23] <pieterh> my use case for PAIR is just interthread signalling
[Wednesday 01 September 2010] [10:44:32] <sustrik> hm
[Wednesday 01 September 2010] [10:44:37] <sustrik> what use case?
[Wednesday 01 September 2010] [10:44:52] <pieterh> i explained it in the guide, end of chapter 2
[Wednesday 01 September 2010] [10:45:00] <pieterh> thread a wants to signal to thread b that it’s done
[Wednesday 01 September 2010] [10:45:05] <pieterh> no scaling required
[Wednesday 01 September 2010] [10:45:10] <sustrik> let me see
[Wednesday 01 September 2010] [10:45:14] <pieterh> no reconnection required
[Wednesday 01 September 2010] [10:45:25] <pieterh> PAIR gives the best semantics afaics
[Wednesday 01 September 2010] [10:47:48] * sustrik is looking at the program
[Wednesday 01 September 2010] [10:48:00] <sustrik> this one: “Thread Coordination” ?
[Wednesday 01 September 2010] [10:51:38] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Wednesday 01 September 2010] [10:51:55] <sustrik> well, there’s no business logic in the example
[Wednesday 01 September 2010] [10:52:09] <sustrik> so it’s hard to say what’s the right socket types to use
[Wednesday 01 September 2010] [10:52:39] <sustrik> it the three components are by definition singletons, then PAIR is appropriate
[Wednesday 01 September 2010] [10:52:54] <sustrik> it say step2 can be parallelised the pipeline would be better
[Wednesday 01 September 2010] [10:53:10] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [10:53:49] <sustrik> the problem i have with pair is that it solves a problem (the accounting vs. HR problem)
[Wednesday 01 September 2010] [10:53:54] <sustrik> but it doesn’t scale
[Wednesday 01 September 2010] [10:54:29] <sustrik> by throwing more boxes on your accounting/HR application you won’t get more power
[Wednesday 01 September 2010] [10:54:43] <sustrik> even thought you paired the two by a PAIR socket
[Wednesday 01 September 2010] [11:17:37] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [11:17:48] Nick
pieterh_ is now known as pieterh.
[Wednesday 01 September 2010] [11:22:08] Quit gebi has left this server (Ping timeout: 276 seconds).
[Wednesday 01 September 2010] [11:29:51] <guido_g> sustrik: latest master shows poll problem again
[Wednesday 01 September 2010] [11:38:58] Quit mrm2m has left this server (Quit: Leaving.).
[Wednesday 01 September 2010] [12:11:55] <ModusPwnens> hello
[Wednesday 01 September 2010] [12:18:26] <ModusPwnens> i have a question about the zmq_msg_size function
[Wednesday 01 September 2010] [12:26:24] <cremes> ModusPwnens: go ahead and ask
[Wednesday 01 September 2010] [12:26:33] <cremes> if someone knows the answer, they’ll respond
[Wednesday 01 September 2010] [12:26:38] <ModusPwnens> ok
[Wednesday 01 September 2010] [12:26:56] <ModusPwnens> so as i have said before, I am using google protocl buffers to encode messages and send them
[Wednesday 01 September 2010] [12:27:29] <ModusPwnens> and on the clientsize, I do message.size(), where message is the size of the object encoded by google protocol buffers
[Wednesday 01 September 2010] [12:27:33] <ModusPwnens> however, on the server side
[Wednesday 01 September 2010] [12:27:41] <ModusPwnens> I use the zmq_msg_size function
[Wednesday 01 September 2010] [12:27:55] <ModusPwnens> but no matter how large the message actually is, zmq_msg_size always gives 36 bytes
[Wednesday 01 September 2010] [12:28:14] <ModusPwnens> I am wondering not only why that is, but how to get the actual size of the payload of the message
[Wednesday 01 September 2010] [12:28:21] <cremes> ok
[Wednesday 01 September 2010] [12:28:48] <cremes> it sounds to me like your client is not allocating the message correctly
[Wednesday 01 September 2010] [12:29:11] <cremes> do you have a code snippet showing how you are allocating the zmq_msg_t and assigning your protocol buffer to it?
[Wednesday 01 September 2010] [12:29:36] <ModusPwnens> rc = zmq_msg_init_data(&msg, cstr, message.size(), NULL, NULL);
[Wednesday 01 September 2010] [12:29:41] <ModusPwnens> something like that?>
[Wednesday 01 September 2010] [12:30:43] <cremes> hold on a sec...
[Wednesday 01 September 2010] [12:30:51] <ModusPwnens> k
[Wednesday 01 September 2010] [12:32:32] <cremes> the variable ‘cstr’ points to the data; are you sure its size is the same as message.size() ?
[Wednesday 01 September 2010] [12:32:47] <Steve-o> shouldn’t it be message.ByteSize() in Google protobufs?
[Wednesday 01 September 2010] [12:33:26] * cremes thinks that is a good question too since i am unfamiliar with that api
[Wednesday 01 September 2010] [12:34:08] <ModusPwnens> hmm, that’s a good point. I wasn’t aware that function existed..
[Wednesday 01 September 2010] [12:34:26] <ModusPwnens> but in any case, that doesn’t seem like it would solve my problem.
[Wednesday 01 September 2010] [12:35:03] <cremes> does the server always receive packets of length 36 exactly? if so, the client must be producing messages of that length
[Wednesday 01 September 2010] [12:35:25] <cremes> focus on the client side for now... something is likely wrong there
[Wednesday 01 September 2010] [12:35:59] <cremes> on the server, are the bytes received garbage? try printing the data the client sends and comparing it to the data the server receives
[Wednesday 01 September 2010] [12:36:10] <ModusPwnens> It seems so, but when i print the value of message.size() in the client, it prints outs omething much larger
[Wednesday 01 September 2010] [12:36:16] <ModusPwnens> and the bytes received are not garbage
[Wednesday 01 September 2010] [12:36:21] <ModusPwnens> i have verified the message coming through is valid
[Wednesday 01 September 2010] [12:37:17] <cremes> why not try: rc = zmq_msg_init_data(&msg, cstr, strlen(cstr), NULL, NULL);
[Wednesday 01 September 2010] [12:37:48] <cremes> i don’t understand the relationship between cstr and message.size()
[Wednesday 01 September 2010] [12:38:21] <ModusPwnens> Hmm, i changed that for a reason...
[Wednesday 01 September 2010] [12:38:32] <ModusPwnens> I think it’s because the zmq syntax doesn’t want a string there or something
[Wednesday 01 September 2010] [12:38:41] <ModusPwnens> so I had to convert a string into a c-style string
[Wednesday 01 September 2010] [12:38:46] <ModusPwnens> and thats what cstr is
[Wednesday 01 September 2010] [12:39:08] <cremes> ah ha... that is your problem
[Wednesday 01 September 2010] [12:39:19] <ModusPwnens> it is?
[Wednesday 01 September 2010] [12:39:20] <cremes> c style strings are terminated by a NUL 0
[Wednesday 01 September 2010] [12:39:50] <ModusPwnens> Yeah. Does that interfere with zeromq somehow?
[Wednesday 01 September 2010] [12:39:59] <cremes> hmmm, wait, i got ahead of myself there...
[Wednesday 01 September 2010] [12:40:14] <cremes> i’m going to think out loud
[Wednesday 01 September 2010] [12:40:17] <ModusPwnens> ok
[Wednesday 01 September 2010] [12:40:31] <cremes> so a c style string is really just a char* or a pointer to a byte buffer
[Wednesday 01 September 2010] [12:40:45] <ModusPwnens> correct
[Wednesday 01 September 2010] [12:40:48] <ModusPwnens> oh right
[Wednesday 01 September 2010] [12:40:51] <cremes> you need to pass a byte buffer pointing to the beginning of your encoded message
[Wednesday 01 September 2010] [12:40:52] <ModusPwnens> sorry you’re thinking out loud >_<
[Wednesday 01 September 2010] [12:41:06] <cremes> is the cstr pointing to that?
[Wednesday 01 September 2010] [12:41:13] <ModusPwnens> yeah
[Wednesday 01 September 2010] [12:41:28] <ModusPwnens> and then i strcpy the message into cstr
[Wednesday 01 September 2010] [12:41:29] <cremes> e.g. cstr = (char*) message.encode().get_pointer()
[Wednesday 01 September 2010] [12:41:30] <ModusPwnens> beforehand
[Wednesday 01 September 2010] [12:41:48] <ModusPwnens> yeah something like that
[Wednesday 01 September 2010] [12:41:49] <cremes> any embedded NULs will terminate the string copy
[Wednesday 01 September 2010] [12:42:06] <cremes> so you may only be copying 36 bytes even though your byte buffer is larger
[Wednesday 01 September 2010] [12:42:10] <cremes> use memcpy instead
[Wednesday 01 September 2010] [12:42:20] <cremes> does that sound reasonable?
[Wednesday 01 September 2010] [12:42:34] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [12:42:38] <ModusPwnens> but..wouldn’t that reflect on the serverside as far as displaying the message?
[Wednesday 01 September 2010] [12:42:47] <ModusPwnens> i.e. it wouldn’t display the whole thing?
[Wednesday 01 September 2010] [12:43:10] <ModusPwnens> furthermore
[Wednesday 01 September 2010] [12:43:18] <ModusPwnens> embedded nuls will terminate the string copy
[Wednesday 01 September 2010] [12:43:29] <ModusPwnens> but doesnt’ that only apply to the encoded message that is being copied?
[Wednesday 01 September 2010] [12:43:47] <cremes> ModusPwnens: you are going to have to show more code, i think
[Wednesday 01 September 2010] [12:44:15] <ModusPwnens> ok i will just copy the entire function into a pastebin
[Wednesday 01 September 2010] [12:44:19] <cremes> there are lots of folks successfully slinging around large strings and large byte buffers so there is likely an issue in your code
[Wednesday 01 September 2010] [12:44:27] <cremes> cool
[Wednesday 01 September 2010] [12:44:32] <ModusPwnens> That is definitely a possibility
[Wednesday 01 September 2010] [12:44:49] <cremes> you are using 2.0.8 release, yes? or the 2.1 master branch?
[Wednesday 01 September 2010] [12:44:58] <ModusPwnens> 2.0.7
[Wednesday 01 September 2010] [12:45:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r99ddfa7 10/ (builds/msvc/platform.hpp configure.in): maint: will become 2.0.9 -
http://bit.ly/dgicJb[Wednesday 01 September 2010] [12:45:26] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47aaf10 10/ :
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: * maint:
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: maint: will become 2.0.9
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: Conflicts:
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: builds/msvc/platform.hpp
[Wednesday 01 September 2010] [12:45:28] <CIA-20> zeromq2: 03Mikael Helbo Kjr 07master * r59315eb 10/ src/select.cpp :
[Wednesday 01 September 2010] [12:45:29] <CIA-20> zeromq2: Erasure of retired fd’s in select.cpp causes an assertion in MSVC 2008 STL
[Wednesday 01 September 2010] [12:45:29] <CIA-20> zeromq2: I was hitting an issue with an SCL enabled STL library in connection with the
[Wednesday 01 September 2010] [12:45:30] <CIA-20> zeromq2: way select_t::loop was erasing retired fd’s. The problem as identified by the
[Wednesday 01 September 2010] [12:45:31] <CIA-20> zeromq2: SCL assertion was that by the time the iterator given to the erase method was
[Wednesday 01 September 2010] [12:45:31] <CIA-20> zeromq2: called it was considered invalid by the library. I am not sure this isn’t just
[Wednesday 01 September 2010] [12:45:32] <CIA-20> zeromq2: a “quirk” of the MSVC STL library as the other code looks valid to me as well. -
http://bit.ly/dzUsD3[Wednesday 01 September 2010] [12:45:32] <CIA-20> zeromq2: 03Martin Sustrik 07master * ra81a373 10/ src/select.cpp :
[Wednesday 01 September 2010] [12:45:33] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 01 September 2010] [12:45:55] <cremes> 2.0.7 should be okay too; i don’t recall any data truncation bugs for any sockets
[Wednesday 01 September 2010] [12:46:35] <ModusPwnens> if i am doing anything stupid in general
[Wednesday 01 September 2010] [12:46:37] <ModusPwnens> please let me know
[Wednesday 01 September 2010] [12:46:59] <ModusPwnens> I am actually only a stupid so im not as experienced as most people here probably are..
[Wednesday 01 September 2010] [12:47:02] <ModusPwnens> wow
[Wednesday 01 September 2010] [12:47:07] <ModusPwnens> i meant student
[Wednesday 01 September 2010] [12:47:35] <cremes> line 36 confuses me...
[Wednesday 01 September 2010] [12:47:41] <ModusPwnens> oh yeah ignore that
[Wednesday 01 September 2010] [12:47:54] <cremes> is that supposed to be commented out?
[Wednesday 01 September 2010] [12:48:27] <cremes> because it is overwriting the size of the msg on the prior line
[Wednesday 01 September 2010] [12:48:37] <ModusPwnens> I think so. We put that in for debugging purposes.
[Wednesday 01 September 2010] [12:48:41] <cremes> with 36 bytes which matches up with how many you are receiving on the server side
[Wednesday 01 September 2010] [12:49:00] <cremes> well, that’s a bug
[Wednesday 01 September 2010] [12:49:06] <cremes> remove that and test again
[Wednesday 01 September 2010] [12:49:13] <ModusPwnens> sigh i hate asking stupid questions
[Wednesday 01 September 2010] [12:51:05] Quit plq has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [12:51:06] <cremes> don’t worry about it; you’ll be repaying the favor here in a few short weeks
[Wednesday 01 September 2010] [12:51:34] <ModusPwnens> ahhhhh gosh darn it
[Wednesday 01 September 2010] [12:51:35] <ModusPwnens> that was it
[Wednesday 01 September 2010] [12:51:41] <ModusPwnens> I’m sorry for wasting your time.
[Wednesday 01 September 2010] [12:51:45] Join jond has joined this channel (
~jon@87.74.16.24).
[Wednesday 01 September 2010] [12:51:45] <ModusPwnens> thanks for your help tho
[Wednesday 01 September 2010] [12:51:46] <cremes> btw, line 34 looks to me like a bug waiting to happen
[Wednesday 01 September 2010] [12:51:51] <cremes> i would use memcpy
[Wednesday 01 September 2010] [12:52:02] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [12:52:27] <ModusPwnens> because of the nul string thing?
[Wednesday 01 September 2010] [12:52:49] <cremes> right
[Wednesday 01 September 2010] [12:53:05] <cremes> memcpy(cstr, message.c_str(), message.size())
[Wednesday 01 September 2010] [12:53:11] <ModusPwnens> Ok, i will change it.
[Wednesday 01 September 2010] [12:53:13] <cremes> or use strncpy
[Wednesday 01 September 2010] [12:53:25] <cremes> but memcpy is likely quicker since it isn’t looking for NULs
[Wednesday 01 September 2010] [12:53:48] <cremes> ModusPwnens: and all this from a Ruby guy who hasn’t used C in anger for many years... :)
[Wednesday 01 September 2010] [12:54:10] <ModusPwnens> heh, well it looks like you still got it then.
[Wednesday 01 September 2010] [12:54:36] <ModusPwnens> oh, and i have another question this time
[Wednesday 01 September 2010] [12:54:48] <ModusPwnens> for the send and receive topology
[Wednesday 01 September 2010] [12:55:10] <ModusPwnens> is there a flag you can set or something that prevents the send and receive messages from blocking?
[Wednesday 01 September 2010] [12:55:54] <cremes> yes
[Wednesday 01 September 2010] [12:56:37] <cremes> btw, when using PUB sockets, it’s a good idea to put your topic in a separate message part
[Wednesday 01 September 2010] [12:57:00] <ModusPwnens> what do you mean?
[Wednesday 01 September 2010] [12:57:12] <cremes> the example code you pastied showed that you are sending the AddressBook as the first message part, so subscribers will match on it
[Wednesday 01 September 2010] [12:57:30] <cremes> i’ll pastie some pseudo-code showing what i mean
[Wednesday 01 September 2010] [12:57:39] <ModusPwnens> ok.
[Wednesday 01 September 2010] [12:58:46] <ModusPwnens> and as far as the no blocking, we want to use it so we can constantly send messages and then receive a message every once in a while, or vice-versa. But would it be better to just use multiple sockets and a publish-subscribe topology instead?
[Wednesday 01 September 2010] [12:58:47] <jond> cremes: if it’s std::string why not use the copy method std::string::copy?
[Wednesday 01 September 2010] [12:59:21] <mikko> do you actually need a copy there?
[Wednesday 01 September 2010] [12:59:58] <ModusPwnens> well i need to store the message I am sending in that string, so I would need to copy it in, right?
[Wednesday 01 September 2010] [13:01:33] <cremes> jond: i don’t know; i’m not a C++ guy but memcpy is pretty easy to figure out for me
[Wednesday 01 September 2010] [13:02:01] <cremes> mikko: you probably need the copy unless you can guarantee the buffer won’t go away until the message is actually sent
[Wednesday 01 September 2010] [13:02:46] <ModusPwnens> Hmm, that makes sense.
[Wednesday 01 September 2010] [13:02:52] <Steve-o> Does it really need to be a string? I used buf.SerializeToArray() and buf.ParseFromArray().
[Wednesday 01 September 2010] [13:03:56] <cremes> ModusPwnens: by sending a topic, your SUB sockets can use it to filter out unwanted messages
[Wednesday 01 September 2010] [13:04:27] <ModusPwnens> Yes that makes sense. I can see the usefulness of that.
[Wednesday 01 September 2010] [13:04:34] <ModusPwnens> And steve-o, you make a good point
[Wednesday 01 September 2010] [13:04:37] <jond> cremes: ok, there is a method on string, so you could do message.copy(cstr,message.size()), but the code id as you say mainly c
[Wednesday 01 September 2010] [13:04:54] <mikko> ModusPwnens: is that C++ or C ?
[Wednesday 01 September 2010] [13:05:06] <cremes> jond: i’m a ruby guy so take my C suggestions with a grain of salt ;)
[Wednesday 01 September 2010] [13:05:13] <ModusPwnens> well, I am using visual studio C++, but I am actually using the C bindings
[Wednesday 01 September 2010] [13:05:23] <mikko> cremes: yeah, true
[Wednesday 01 September 2010] [13:05:28] <mikko> on the earlier
[Wednesday 01 September 2010] [13:05:28] <ModusPwnens> mostly because I was having trouble with the C++ ones
[Wednesday 01 September 2010] [13:06:28] <ModusPwnens> and steve-o, that would probably be a better/more efficient way of doing this, right?
[Wednesday 01 September 2010] [13:06:43] <ModusPwnens> then I wouldn’t have to do all the c-string manipulation
[Wednesday 01 September 2010] [13:07:10] <Steve-o> it should be the best as of v2.1, but I think Google has changed their api a few times since
[Wednesday 01 September 2010] [13:09:06] <Steve-o> noting ByteSize() is only valid after you have actually filled in values for protobufs, it has caught me a few times
[Wednesday 01 September 2010] [13:09:24] <ModusPwnens> you mean after you have encoded a message?
[Wednesday 01 September 2010] [13:10:28] <Steve-o> I needed the message size before serializing
[Wednesday 01 September 2010] [13:11:28] <ModusPwnens> hmm
[Wednesday 01 September 2010] [13:11:32] <ModusPwnens> for the serializetoarray method
[Wednesday 01 September 2010] [13:11:49] Join gebi has joined this channel (
~gebi@84.119.80.149).
[Wednesday 01 September 2010] [13:12:10] <ModusPwnens> you would need to get the size of the data you are serializing before you can serialize it..?
[Wednesday 01 September 2010] [13:13:54] <Steve-o> For my code I need to reserve space in a socket buffer for the serialized form, so I need the size. Obviously usage case varies for everyone.
[Wednesday 01 September 2010] [13:14:43] <ModusPwnens> You mean the size is always known beforehand? What if you don’t know the size before, can you still use that?
[Wednesday 01 September 2010] [13:15:14] <Steve-o> probably if you know the protobuf object is always going to be smaller than your buffer
[Wednesday 01 September 2010] [13:15:45] <ModusPwnens> Hmm. okay.
[Wednesday 01 September 2010] [13:15:57] <Steve-o> it’s a bit safer to check the size before hand, much like vasprintf()
[Wednesday 01 September 2010] [13:19:44] <ModusPwnens> Yeah that makes sense.
[Wednesday 01 September 2010] [13:19:49] <ModusPwnens> Ok, thanks for your help guys!
[Wednesday 01 September 2010] [13:20:17] <ModusPwnens> I will probably be back again in a few days with something else..hopefully something less stupid.
[Wednesday 01 September 2010] [13:53:25] Quit xla has left this server (Quit: leaving).
[Wednesday 01 September 2010] [13:55:34] <pieterh> folks: if anyone wants to review
http://www.zeromq.com, feedback would be welcome
[Wednesday 01 September 2010] [13:55:45] <pieterh> this is meant to be the welcome site for new 0MQ users
[Wednesday 01 September 2010] [13:56:10] <pieterh> next I’ll remove a lot of the fluff from the zeromq.org front page and menus
[Wednesday 01 September 2010] [14:00:50] <pieterh> my idea is to make the zeromq.org site an index of all projects
[Wednesday 01 September 2010] [14:01:00] <pieterh> kind of ranked by activity somehow
[Wednesday 01 September 2010] [14:05:50] <pieterh> no-one minds? :-)
[Wednesday 01 September 2010] [14:09:03] Quit pieterh has left this server (Read error: Operation timed out).
[Wednesday 01 September 2010] [14:12:16] <Steve-o> inproc is still a new word
[Wednesday 01 September 2010] [14:13:32] <guido_g> the onhover bold is a little overdone
[Wednesday 01 September 2010] [14:15:46] Quit alfborge has left this server (Quit: leaving).
[Wednesday 01 September 2010] [14:16:00] <guido_g> on the page linked from 04READ should show the canonical doku first (the guide & the manual)
[Wednesday 01 September 2010] [14:16:56] <jond> steve-o: agree about inproc, not widely used AFAIK
[Wednesday 01 September 2010] [14:23:39] Quit pieterh has left this server (Ping timeout: 258 seconds).
[Wednesday 01 September 2010] [15:45:07] <mcxx> I got freshly cloned zeromq2 and pyzmq repos from github
[Wednesday 01 September 2010] [16:00:54] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [16:27:03] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 01 September 2010] [16:46:14] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Wednesday 01 September 2010] [17:27:20] Quit jond has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [17:57:57] Quit jsimmons has left this server (Client Quit).
[Wednesday 01 September 2010] [18:27:24] Quit AndrewBC has left this server (Ping timeout: 260 seconds).
[Wednesday 01 September 2010] [18:38:18] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [18:42:55] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Wednesday 01 September 2010] [19:05:24] Quit plq has left this server (Ping timeout: 258 seconds).
[Wednesday 01 September 2010] [19:18:54] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [19:30:09] Quit plq has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [20:03:27] <ModusPwnens> Hi, I’m having an issue with the first byte of a message being corrupted when it is sent. Has anyone experienced this?
[Wednesday 01 September 2010] [20:05:43] <bgranger> mcxx: You need to grab the stable 2.0.7 releases of pyzmq and zeromq.
[Wednesday 01 September 2010] [20:06:02] <bgranger> Trunk of pyzmq works with stable 2.0.8 of zeromq.
[Wednesday 01 September 2010] [20:06:15] <bgranger> We have not started following the post 2.0.8 master of zeromq
[Wednesday 01 September 2010] [20:06:44] <ModusPwnens> clarification, it’s actually the first two bytes that get corrupted
[Wednesday 01 September 2010] [20:07:40] Quit MJW has left this server (Client Quit).
[Wednesday 01 September 2010] [20:08:42] Quit ModusPwnens has left this server.
[Wednesday 01 September 2010] [20:47:00] Quit
mw__ has left this server (Quit: Alea iactus est.).
[Wednesday 01 September 2010] [22:08:48] Quit miguelito has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [22:40:34] <lestrrat> I assume this is the right place to talk about zeromq?
[Wednesday 01 September 2010] [22:42:31] <lestrrat> I just wanted to make if zeromq’s recv() is supposed to bail out properly (or not) when it receives a signal while it’s waiting for a message?
[Wednesday 01 September 2010] [22:42:49] <lestrrat> my perl binding seems to be stuck, and I’m trying to figure out if it’s my problem or not.
[Wednesday 01 September 2010] [23:46:46] <bgranger> lestrrat: I think I know what is going on.
[Wednesday 01 September 2010] [23:46:58] <bgranger> Are you still around?
[Wednesday 01 September 2010] [23:47:02] <lestrrat> yes!
[Wednesday 01 September 2010] [23:47:13] <bgranger> Hey, we have the same issue in the Python bindings.
[Wednesday 01 September 2010] [23:47:27] <bgranger> It perl is like Python, then it sets its own signal handlers.
[Wednesday 01 September 2010] [23:47:33] <lestrrat> ah, so glad I’m not alone on this :)
[Wednesday 01 September 2010] [23:47:34] <lestrrat> right
[Wednesday 01 September 2010] [23:47:47] <bgranger> BUT, while a blocking recv is waiting, perl is completely out of control.
[Wednesday 01 September 2010] [23:48:22] <AndrewBC> I wrapped mine in a try except in python, excepting a KeyboardInterrupt
[Wednesday 01 September 2010] [23:48:25] <AndrewBC> worked fine
[Wednesday 01 September 2010] [23:48:42] <bgranger> SO what happens is that Perl’s signal handler will receive the signal, but it won’t be handled until the recv returns. To test if this is what is really going on, try to create a simple example , then send the signal and send an additional msg to the socket so that recv returns.
[Wednesday 01 September 2010] [23:48:44] <AndrewBC> so you might have to manually catch ^C if that’s possible in perl
[Wednesday 01 September 2010] [23:49:14] <bgranger> AndrewBC, it won’t work if there is not a msg recv’d
[Wednesday 01 September 2010] [23:49:28] <bgranger> Or I should say, the KeyboardInt won’t be raised until recv returns.
[Wednesday 01 September 2010] [23:49:37] <lestrrat> bgranger: ah right
[Wednesday 01 September 2010] [23:49:37] <AndrewBC> oh, hm
[Wednesday 01 September 2010] [23:49:38] <bgranger> You can’t really interupt it with SIGINT
[Wednesday 01 September 2010] [23:50:22] <lestrrat> I can go and confirm, but if that’s the case, it’s really zeromq’s responsibility to return on a signal, no? (with errno = EINTER or whatever)
[Wednesday 01 September 2010] [23:50:35] <bgranger> But the same is true when Python calls any external C/C++ code.
[Wednesday 01 September 2010] [23:50:40] <lestrrat> right
[Wednesday 01 September 2010] [23:50:55] <bgranger> No, because Python’s signal handler is still in place, it just doesn’t get to run.
[Wednesday 01 September 2010] [23:51:20] <lestrrat> yeah, but we can propagate the signal as long as we get back control
[Wednesday 01 September 2010] [23:51:58] <bgranger> But you can’t get back control
[Wednesday 01 September 2010] [23:52:02] <lestrrat> hmm?
[Wednesday 01 September 2010] [23:52:26] <bgranger> The signal handler is the one at the C level, not the one that you can control using the signal module
[Wednesday 01 September 2010] [23:52:41] <lestrrat> yeah, but I’m writing the Perl<->C binding, so I have control :)
[Wednesday 01 September 2010] [23:53:01] <bgranger> To change the behavior you actually to have use signal.h and set jump/long jump logic.
[Wednesday 01 September 2010] [23:53:07] <bgranger> But it can be done
[Wednesday 01 September 2010] [23:54:00] <bgranger> It is just subtle logic and hard to get right in a cross platform manner.
[Wednesday 01 September 2010] [23:54:15] <bgranger> I don’t think any of the language bindings are doing this yet, but eventually all should
[Wednesday 01 September 2010] [23:54:46] <lestrrat> I dunno the python internals, but I don’t think we need a jump logic for perl
[Wednesday 01 September 2010] [23:54:49] <bgranger> Meanwhile, we are just not using blocking recv when it needs to be interupted
[Wednesday 01 September 2010] [23:55:32] <bgranger> If perl sets its own signal handlers at the C level, you likely will.
[Wednesday 01 September 2010] [23:55:35] <lestrrat> perl’s signal handling doesn’t run on “real time”. it’s deferred until the next perl op. so as long as I’m in C land, I can tell the Perl interpreter that there was a signal, and properly emulate it.
[Wednesday 01 September 2010] [23:56:33] <bgranger> That is how Python works as well and it why the default signal handler doesn’t actually interrupt “in real time:”
[Wednesday 01 September 2010] [23:56:58] <lestrrat> then I don’t see why you need a jump...? what am I missing?
[Wednesday 01 September 2010] [23:57:46] <bgranger> I don’t remember the details honestly. I think the issue is that you have to be very careful about restoring Python’s regular signal handler and putting the intepreter back in the right place in the stack.
[Wednesday 01 September 2010] [23:57:57] <lestrrat> hmm.
[Wednesday 01 September 2010] [23:58:25] <bgranger> That is, if you want control to return to Perl/Python in a way that doesn’t simply kill the process.
[Wednesday 01 September 2010] [23:58:38] <lestrrat> actually, I don’t really care either way – I don’t have to have zmq_recv to get back to the Perl handler – I just need it to get back to me once a signal is sent w/o killing the process
[Wednesday 01 September 2010] [23:58:45] <lestrrat> otherwise servers hang.
[Wednesday 01 September 2010] [23:59:01] <bgranger> yep
[Wednesday 01 September 2010] [23:59:14] <lestrrat> glad to know I’m the only one. I can safely write an RFE :)
[Thursday 02 September 2010] [00:43:40] <sustrik> bgranger, lestrrat: i am still not sure about EINTR behaviour
[Thursday 02 September 2010] [00:43:53] <sustrik> would returning EINTR from blocking call help in any way?
[Thursday 02 September 2010] [00:47:53] <bgranger> In what situations?
[Thursday 02 September 2010] [00:49:02] <bgranger> Are you thinking of installing a signal handler that catches SIGINT and translates that into EINTR?
[Thursday 02 September 2010] [00:49:33] <bgranger> That would help us, but that signal handler would have to play nice with that of Python
[Thursday 02 September 2010] [00:52:11] <sustrik> bgragner: no, i meant the peculiar functionality of 0mq
[Thursday 02 September 2010] [00:52:25] <sustrik> that catches OS’s EINTR
[Thursday 02 September 2010] [00:52:34] <sustrik> and instead of returning it to the user
[Thursday 02 September 2010] [00:52:45] <sustrik> just restarts the blocking operation
[Thursday 02 September 2010] [00:53:01] <bgranger> I am not familiar with what zmq is doing right now
[Thursday 02 September 2010] [00:53:07] <sustrik> but afaiu that’s not the problem you are dealing with
[Thursday 02 September 2010] [00:53:30] <bgranger> When does the OS signal EINTR?
[Thursday 02 September 2010] [00:53:47] <bgranger> And how does zeromq currently handle it?
[Thursday 02 September 2010] [00:54:00] <sustrik> if a threads gets a signal while in blocking operation
[Thursday 02 September 2010] [00:54:33] <sustrik> 0mq currently ignores it, i.e. restarts the blocking op
[Thursday 02 September 2010] [00:56:29] <bgranger> I guess what I don’t know if how all of this interplays with Python’s signal handling.
[Thursday 02 September 2010] [00:56:35] <sustrik> exactly
[Thursday 02 September 2010] [00:56:43] <bgranger> Python only receives signals in the main thread
[Thursday 02 September 2010] [00:56:55] <bgranger> Does zeromq install any signal handlers?
[Thursday 02 September 2010] [00:56:57] <sustrik> that’s the one that’s blocked in 0mq?
[Thursday 02 September 2010] [00:57:05] <sustrik> no, it does not
[Thursday 02 September 2010] [00:57:10] <bgranger> OK
[Thursday 02 September 2010] [00:57:21] <bgranger> but when a socket call returns EINTR, it just continues
[Thursday 02 September 2010] [00:57:28] <sustrik> right
[Thursday 02 September 2010] [00:57:34] <sustrik> actaully, you can check what’s happening yourself
[Thursday 02 September 2010] [00:57:37] <bgranger> OK, then another ?
[Thursday 02 September 2010] [00:57:41] <bgranger> OK
[Thursday 02 September 2010] [00:57:46] <bgranger> How...
[Thursday 02 September 2010] [00:57:46] <sustrik> when waiting for a message
[Thursday 02 September 2010] [00:57:59] <sustrik> 0mq is stuck in a read call
[Thursday 02 September 2010] [00:58:06] <bgranger> Correct
[Thursday 02 September 2010] [00:58:09] <sustrik> let me find the exact code...
[Thursday 02 September 2010] [00:58:18] <bgranger> OK great, that would be helpful.
[Thursday 02 September 2010] [00:59:27] <bgranger> Are the other occasions where EINTR is returned?
[Thursday 02 September 2010] [01:00:06] <sustrik> polling
[Thursday 02 September 2010] [01:00:12] <sustrik> are you on linux?
[Thursday 02 September 2010] [01:00:16] <lestrrat> sorry was away
[Thursday 02 September 2010] [01:00:21] <bgranger> linux and Mac
[Thursday 02 September 2010] [01:00:22] <sustrik> hi
[Thursday 02 September 2010] [01:00:31] <bgranger> Talking about signal handling more...
[Thursday 02 September 2010] [01:00:37] <sustrik> trunk or 2.0.8?
[Thursday 02 September 2010] [01:00:55] <bgranger> pyzmq doesn’t work with trunk yet, sso 2.0.8 or 2.0.7
[Thursday 02 September 2010] [01:00:58] <lestrrat> I’m using 2.0.8
[Thursday 02 September 2010] [01:01:53] <bgranger> but if we can do signal -> EINTR -> Python bindings -> raise KeyboardInterrupt that would be great
[Thursday 02 September 2010] [01:02:04] <sustrik> yes
[Thursday 02 September 2010] [01:02:14] <sustrik> you see the loop, right?
[Thursday 02 September 2010] [01:02:21] <lestrrat> right
[Thursday 02 September 2010] [01:02:28] <bgranger> Yep
[Thursday 02 September 2010] [01:02:33] <sustrik> just put a printf there or something
[Thursday 02 September 2010] [01:02:42] <sustrik> and check whether it loops in case of SIGINT
[Thursday 02 September 2010] [01:02:48] <bgranger> OK I will do that
[Thursday 02 September 2010] [01:02:56] <bgranger> I will check on Linux and OS X
[Thursday 02 September 2010] [01:02:57] <sustrik> if so we can return EINTR instead of looping
[Thursday 02 September 2010] [01:03:00] <sustrik> great
[Thursday 02 September 2010] [01:03:01] <sustrik> thanks
[Thursday 02 September 2010] [01:03:04] <bgranger> Cool
[Thursday 02 September 2010] [01:03:07] <bgranger> Thanks!
[Thursday 02 September 2010] [01:03:15] <bgranger> Won’t get to it tonight though
[Thursday 02 September 2010] [01:03:21] <bgranger> Getting late here...
[Thursday 02 September 2010] [01:03:39] <sustrik> sure
[Thursday 02 September 2010] [01:03:53] <sustrik> lestrrat: you may try the same, if in hurry
[Thursday 02 September 2010] [01:03:55] <lestrrat> yeah, I’m on $day_job too, so I’m not going to be able to check it right away. Will see if I can sneak that in in the next couple of hours
[Thursday 02 September 2010] [01:04:01] <bgranger> If I want to try the change, I can just have the function return EINTR?
[Thursday 02 September 2010] [01:04:11] <lestrrat> yep, will do when pointy haired boss isn’t looking
[Thursday 02 September 2010] [01:04:17] <sustrik> :)
[Thursday 02 September 2010] [01:04:18] <lestrrat> AFK for now
[Thursday 02 September 2010] [01:04:28] <bgranger> Will the return value of that function be returned to the caller
[Thursday 02 September 2010] [01:04:33] <sustrik> bgranger: probably
[Thursday 02 September 2010] [01:04:46] <sustrik> let me check
[Thursday 02 September 2010] [01:04:52] <bgranger> Well, it looks like it must return a bool though
[Thursday 02 September 2010] [01:05:07] <bgranger> May have to change more logic.
[Thursday 02 September 2010] [01:05:16] <sustrik> ah, ok
[Thursday 02 September 2010] [01:05:24] <sustrik> it’s not propagated
[Thursday 02 September 2010] [01:05:30] <sustrik> but i can fix that
[Thursday 02 September 2010] [01:05:36] <sustrik> the question is whether it would help
[Thursday 02 September 2010] [01:05:38] <bgranger> OK, let’s test first
[Thursday 02 September 2010] [01:05:40] <bgranger> right
[Thursday 02 September 2010] [01:09:00] <sustrik> lestrrat: are you in japan? i’ve notices some of your twitter notes
[Thursday 02 September 2010] [01:09:12] <sustrik> had no idea what they were about though :)
[Thursday 02 September 2010] [01:10:29] <guido_g> hi all
[Thursday 02 September 2010] [01:10:37] <guido_g> good morning sustrik
[Thursday 02 September 2010] [01:10:44] <sustrik> morning
[Thursday 02 September 2010] [01:11:11] <guido_g> did you read that the poll segfault in master is still there?
[Thursday 02 September 2010] [01:11:38] <sustrik> yep, seen you saying it yesterday
[Thursday 02 September 2010] [01:11:42] <sustrik> can i reproduce it?
[Thursday 02 September 2010] [01:11:50] <guido_g> i hope so
[Thursday 02 September 2010] [01:11:58] <sustrik> which test program?
[Thursday 02 September 2010] [01:12:28] <guido_g> zmqcpp and sender
[Thursday 02 September 2010] [01:12:42] <guido_g> zmqcpp segfaults in poll
[Thursday 02 September 2010] [01:13:01] <sustrik> what order to start them in?
[Thursday 02 September 2010] [01:13:20] <guido_g> i started zmqcpp first and then sender
[Thursday 02 September 2010] [01:13:26] <guido_g> sender is a one shot thing
[Thursday 02 September 2010] [01:13:30] <sustrik> ok
[Thursday 02 September 2010] [01:14:18] <guido_g> going to write a mq test program for that
[Thursday 02 September 2010] [01:14:24] <sustrik> ack
[Thursday 02 September 2010] [01:14:44] <guido_g> seems to be a nasty corner that needs some regression
[Thursday 02 September 2010] [01:22:52] <sustrik> guido_g:
[Thursday 02 September 2010] [01:22:54] <sustrik> In file included from zmqcpp.cpp:9:
[Thursday 02 September 2010] [01:22:54] <sustrik> worker.hpp:14:21: error: pqxx/pqxx: No such file or directory
[Thursday 02 September 2010] [01:22:56] <sustrik> ...
[Thursday 02 September 2010] [01:23:05] <guido_g> oh shit
[Thursday 02 September 2010] [01:23:07] <guido_g> sorry
[Thursday 02 September 2010] [01:23:14] <sustrik> I’ll rather wait for your test program
[Thursday 02 September 2010] [01:23:21] <guido_g> ack
[Thursday 02 September 2010] [01:23:29] <guido_g> never mind then
[Thursday 02 September 2010] [01:23:45] <guido_g> “i’ll be back!” :)
[Thursday 02 September 2010] [01:24:07] <sustrik> ;)
[Thursday 02 September 2010] [01:24:45] <guido_g> so, day job is waiting, cu
[Thursday 02 September 2010] [01:24:57] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [01:31:01] <lestrrat> sustrik: yes I’m in Japan :)
[Thursday 02 September 2010] [01:32:01] <sustrik> lot of japanese tweets about 0mq lately
[Thursday 02 September 2010] [01:32:04] <sustrik> it’s fun
[Thursday 02 September 2010] [01:33:44] <lestrrat> let’s see if I can work on that signal thing...
[Thursday 02 September 2010] [01:34:11] <sustrik> ok
[Thursday 02 September 2010] [01:40:09] <lestrrat> alright, so I’m a autoconf noob. how do I make this go away?:
[Thursday 02 September 2010] [01:40:16] Notice -lestrrat to #zeromq- ./configure: line 18331: syntax error near unexpected token
`GLIB,’
[Thursday 02 September 2010] [01:40:17] Notice -lestrrat to #zeromq- ./configure: line 18331: ` PKG_CHECK_MODULES(GLIB, glib-2.0 gthread-2.0)’
[Thursday 02 September 2010] [01:41:27] <sustrik> you need newer version of autoconf
[Thursday 02 September 2010] [01:41:34] <sustrik> are you on OSX?
[Thursday 02 September 2010] [01:42:40] <sustrik> this one would deserve going into FAQ btw
[Thursday 02 September 2010] [01:43:01] <lestrrat> yeah, I’m on OSX
[Thursday 02 September 2010] [01:43:18] <sustrik> it’s a known issue
[Thursday 02 September 2010] [01:43:22] <lestrrat> grrr
[Thursday 02 September 2010] [01:43:30] * lestrrat looks for autoconf in homebrew
[Thursday 02 September 2010] [01:43:34] <sustrik> osx has old version of autoconf installed by default
[Thursday 02 September 2010] [01:44:26] <lestrrat> grr, and it’s not there in homebrew
[Thursday 02 September 2010] [01:47:35] <lestrrat> man, this is going to take some time :)
[Thursday 02 September 2010] [01:47:37] <sustrik> lestrrat: sorry, it’s pkg-config
[Thursday 02 September 2010] [01:47:55] * sustrik is autotools noob as well
[Thursday 02 September 2010] [01:47:58] <lestrrat> oh cool
[Thursday 02 September 2010] [01:48:52] <lestrrat> hmm, I seem to have the latest pkg-config
[Thursday 02 September 2010] [01:49:25] <sustrik> are you building witg –with-pgm btw?
[Thursday 02 September 2010] [01:50:15] <lestrrat> nay, just the default with a custom –prefix
[Thursday 02 September 2010] [01:50:19] <sustrik> hm
[Thursday 02 September 2010] [01:50:28] <sustrik> well, you’ll have to ask on the mailing list
[Thursday 02 September 2010] [01:50:38] <sustrik> as i say, i’m an autotools idiot
[Thursday 02 September 2010] [01:53:06] <lestrrat> trying out autoconf anyways
[Thursday 02 September 2010] [02:09:07] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rd5b6f68 10/ AUTHORS : Mikael Kjaer added to AUTHORS -
http://bit.ly/aI6jV2[Thursday 02 September 2010] [02:09:08] <CIA-20> zeromq2: 03Bernd Melchers 07maint * r8ec0743 10/ (AUTHORS src/signaler.cpp): Fix for signaler_t on HP-UX and AIX platforms -
http://bit.ly/9XM0R0[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: 03Jon Dyte 07maint * r14853c2 10/ (src/prefix_tree.cpp src/prefix_tree.hpp):
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: This is because as it adds the 255th element at a node it attempts to calculate
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: the count member var which is an unsigned char via count = (255 -0) + 1; and
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: pass the result to realloc. Unfortunately the result is zero and realloc returns
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: null; the prefix_tree asserts. I have fixed it by making the count an unsigned
[Thursday 02 September 2010] [02:09:35] <CIA-20> zeromq2: 03Martin Sustrik 07master * rd5b6f68 10/ AUTHORS : Mikael Kjaer added to AUTHORS -
http://bit.ly/aI6jV2[Thursday 02 September 2010] [02:09:36] <CIA-20> zeromq2: 03Bernd Melchers 07master * r8ec0743 10/ (AUTHORS src/signaler.cpp): Fix for signaler_t on HP-UX and AIX platforms -
http://bit.ly/9XM0R0[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: 03Jon Dyte 07master * r14853c2 10/ (src/prefix_tree.cpp src/prefix_tree.hpp):
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: This is because as it adds the 255th element at a node it attempts to calculate
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: the count member var which is an unsigned char via count = (255 -0) + 1; and
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: pass the result to realloc. Unfortunately the result is zero and realloc returns
[Thursday 02 September 2010] [02:09:40] <CIA-20> zeromq2: null; the prefix_tree asserts. I have fixed it by making the count an unsigned
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0a1f7e3 10/ (AUTHORS src/signaler.cpp src/trie.cpp src/trie.hpp): (log message trimmed)
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Merge branch ‘maint’
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: * maint:
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Fix for signaler_t on HP-UX and AIX platforms
[Thursday 02 September 2010] [02:10:04] <CIA-20> zeromq2: Mikael Kjaer added to AUTHORS
[Thursday 02 September 2010] [02:10:04] <CIA-20> zeromq2: Conflicts:
[Thursday 02 September 2010] [02:16:49] Quit bgranger has left this server (Quit: bgranger).
[Thursday 02 September 2010] [02:35:56] <lestrrat> hmm, zeromq HEAD against my perl binding doesn’t work – (seeing some tests block on odd places)
[Thursday 02 September 2010] [02:36:02] <lestrrat> could be something on my part, of course.
[Thursday 02 September 2010] [02:36:05] <lestrrat> investigating...
[Thursday 02 September 2010] [02:52:46] <lestrrat> hmm, switching back to 2.0.8 makes the problem go away. I see all my destructors running, so I’m willing to be there’s some sort of background thread or something that’s kept alive in zeromq
[Thursday 02 September 2010] [02:54:03] <guido_g> re
[Thursday 02 September 2010] [02:58:55] <sustrik> lestrrat: HEAD is a dev version, testing with 2.0.8 is ok
[Thursday 02 September 2010] [02:59:03] <lestrrat> k
[Thursday 02 September 2010] [02:59:12] <sustrik> what about the SIGINT thing?
[Thursday 02 September 2010] [02:59:49] <lestrrat> trying now
[Thursday 02 September 2010] [03:02:10] <lestrrat> so yeah, looks like it’s stuck in that loop
[Thursday 02 September 2010] [03:02:29] <lestrrat> tweaking to see if I can cleanly exit there...
[Thursday 02 September 2010] [03:05:37] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [03:11:16] <sustrik> lestrrat: i’m going to be away now for a while, but leave me a message here about whether returning EINTR from 0mq would help you
[Thursday 02 September 2010] [03:11:37] <lestrrat> yep, thanks
[Thursday 02 September 2010] [03:23:34] <lestrrat> this works for me!
[Thursday 02 September 2010] [03:41:43] Quit grahamalot has left this server (Quit: grahamalot).
[Thursday 02 September 2010] [04:30:52] Quit pieterh has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [04:45:31] <sustrik> lestrrat: great
[Thursday 02 September 2010] [04:45:51] <sustrik> i’ll fix it in a bit different way (actually returning EINTR)
[Thursday 02 September 2010] [04:46:04] <sustrik> but it’s good news to know it solves the Ctrl+C problem
[Thursday 02 September 2010] [04:46:40] <sustrik> we’ll see whether it helps with python as well
[Thursday 02 September 2010] [04:52:42] Quit gebi has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [04:58:39] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 02 September 2010] [05:00:31] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 02 September 2010] [05:19:30] Quit AndrewBC has left this server (Ping timeout: 258 seconds).
[Thursday 02 September 2010] [05:21:08] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [05:23:35] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [05:23:42] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 02 September 2010] [05:52:32] <mato> sustrik: are you there?
[Thursday 02 September 2010] [06:00:55] <sssss> test
[Thursday 02 September 2010] [06:01:15] Quit sssss has left this server (Client Quit).
[Thursday 02 September 2010] [06:06:57] <sustrik> mato: re
[Thursday 02 September 2010] [06:14:51] <mato> sustrik: could you switch on all the test servers + switch please?
[Thursday 02 September 2010] [06:15:15] <mato> sustrik: actually, just one of the 8-core boxes is all i need right now
[Thursday 02 September 2010] [06:15:32] <mato> sustrik: and the two white ones
[Thursday 02 September 2010] [06:15:51] <mato> sustrik: so pick the quieter of janos or csaba :) i don’t care...
[Thursday 02 September 2010] [06:16:23] <sustrik> oh my
[Thursday 02 September 2010] [06:16:24] <sustrik> ok
[Thursday 02 September 2010] [06:16:41] * sustrik is inserting earplugs
[Thursday 02 September 2010] [06:16:46] <mato> :)
[Thursday 02 September 2010] [06:16:59] <mato> loud music also works :)
[Thursday 02 September 2010] [06:18:13] <sustrik> started
[Thursday 02 September 2010] [06:18:30] <mato> damn
[Thursday 02 September 2010] [06:18:31] <sustrik> now i realise there’s just one of those big boxes here
[Thursday 02 September 2010] [06:18:38] <sustrik> where’s the other one?
[Thursday 02 September 2010] [06:18:39] <pieterh> hey, the lights just dimmed here...!
[Thursday 02 September 2010] [06:18:50] <sustrik> still at malosek’s place?
[Thursday 02 September 2010] [06:18:52] <mato> sustrik: just one? i dunno? does malo still have the other?
[Thursday 02 September 2010] [06:19:02] <mato> sustrik: i can’t connect to your IP at all :(
[Thursday 02 September 2010] [06:19:22] <mato> hmm
[Thursday 02 September 2010] [06:19:29] <sustrik> i see the world outside
[Thursday 02 September 2010] [06:19:55] <mato> i have a suspicion your chello connection is funny
[Thursday 02 September 2010] [06:20:08] <sustrik> ?
[Thursday 02 September 2010] [06:20:22] <mato> well, first of all, your ip is on some weird subnet
[Thursday 02 September 2010] [06:20:42] <mato> which i’ve never seen
[Thursday 02 September 2010] [06:20:49] <mato> 2nd, i can’t ping or connect to it :(
[Thursday 02 September 2010] [06:21:11] <sustrik> :(
[Thursday 02 September 2010] [06:21:31] <sustrik> you can take the boxes
[Thursday 02 September 2010] [06:34:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [06:44:06] <guido_g> openvpn is really nice for this kind of things
[Thursday 02 September 2010] [06:45:44] <sustrik> it works now (not sure why)
[Thursday 02 September 2010] [07:10:56] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 02 September 2010] [07:24:02] Quit pieterh has left this server (Ping timeout: 272 seconds).
[Thursday 02 September 2010] [07:26:13] <mato> wow
[Thursday 02 September 2010] [07:26:19] <mato> sustrik: quick question...
[Thursday 02 September 2010] [07:26:40] <mato> sustrik: zmq_free_fn() is called in the context of the application or i/o thread?
[Thursday 02 September 2010] [07:27:24] <sustrik> i/o
[Thursday 02 September 2010] [07:27:28] <mato> riiiiiight
[Thursday 02 September 2010] [07:27:33] <mato> that explains a lot
[Thursday 02 September 2010] [07:27:48] <sustrik> ?
[Thursday 02 September 2010] [07:28:05] <mato> i’m having issues with the python GIL
[Thursday 02 September 2010] [07:28:18] <mato> at least i think that’s what happening
[Thursday 02 September 2010] [07:28:24] <sustrik> a-ha, i see
[Thursday 02 September 2010] [07:28:31] <mato> it explains all the mysterious hanging close() with 2.0.x
[Thursday 02 September 2010] [07:28:38] <sustrik> python zero-copy functionality
[Thursday 02 September 2010] [07:28:43] <mato> ja
[Thursday 02 September 2010] [07:28:59] <mato> anyway, i’m going to lunch, we can discuss when i get back...
[Thursday 02 September 2010] [07:29:03] <sustrik> ok
[Thursday 02 September 2010] [07:38:29] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Thursday 02 September 2010] [07:47:22] <sustrik> thanks
[Thursday 02 September 2010] [07:47:37] <sustrik> have you thought of a pub to meet the other guys in?
[Thursday 02 September 2010] [07:47:50] <sustrik> it’s only 3 people so far
[Thursday 02 September 2010] [07:47:58] <sustrik> so no need for complex arrangements
[Thursday 02 September 2010] [07:48:18] <sustrik> say it’ll be 5-7 ultimately
[Thursday 02 September 2010] [07:48:49] <sustrik> oops :)
[Thursday 02 September 2010] [07:48:57] <sustrik> that was intended for another window
[Thursday 02 September 2010] [07:49:03] <sustrik> the london meetup stuff
[Thursday 02 September 2010] [08:16:57] <mikko> sustrik: theodore bullfrog is a decent place
[Thursday 02 September 2010] [08:18:58] <sustrik> looks central enough
[Thursday 02 September 2010] [08:20:46] <sustrik> is that restaurant or a pub?
[Thursday 02 September 2010] [08:24:12] <sustrik> mikko: hullo!
[Thursday 02 September 2010] [08:24:53] <mikko> pub
[Thursday 02 September 2010] [08:24:57] <mikko> but they serve food as well
[Thursday 02 September 2010] [08:25:27] <sustrik> ok, why not then
[Thursday 02 September 2010] [08:25:47] Quit guido_g has left this server (Disconnected by services).
[Thursday 02 September 2010] [08:25:59] Nick OldNugget is now known as guido_g.
[Thursday 02 September 2010] [08:26:31] <sustrik> nice
[Thursday 02 September 2010] [08:36:17] <mato> sustrik: ok, so this GIL thing
[Thursday 02 September 2010] [08:36:24] <mato> sustrik: it’s an interesting problem
[Thursday 02 September 2010] [08:37:13] <sustrik> yes?
[Thursday 02 September 2010] [08:37:24] <mato> what i see is that in my python application zmq_close() is getting called, due to Python gc kicking in
[Thursday 02 September 2010] [08:37:57] <mato> however, at the same time, zmq_msg_close() on a method shared with Python is called in the i/o thread
[Thursday 02 September 2010] [08:38:35] <mato> now, zmq_close() calls process_commands() which is waiting for a signal from the i/o thread
[Thursday 02 September 2010] [08:39:00] <mato> but that signal never arrives, because the free_fn() called by the zmq_msg_close() in the i/o thread is trying to acquire the Python GIL
[Thursday 02 September 2010] [08:39:05] <mato> -> deadlock
[Thursday 02 September 2010] [08:39:47] <mato> you follow me?
[Thursday 02 September 2010] [08:40:11] <mato> anyway, i can work around it now by not closing yet more sockets :)
[Thursday 02 September 2010] [08:40:17] <mato> but it’s an interesting case
[Thursday 02 September 2010] [08:40:35] <mato> i will have to make a test case and write it up on the mailing list so that other people can participate
[Thursday 02 September 2010] [08:40:44] <mato> it’s not clear what the solution is
[Thursday 02 September 2010] [08:40:48] <sustrik> i have to draw it down :)
[Thursday 02 September 2010] [08:41:02] <sustrik> mato: yes, please
[Thursday 02 September 2010] [08:41:06] <sustrik> so that it’s not lost
[Thursday 02 September 2010] [08:41:09] <mato> definitely
[Thursday 02 September 2010] [08:41:18] <mato> it needs to be solved, and i suspect the same problem can occur in 2.1
[Thursday 02 September 2010] [08:41:34] <mato> however, it’s possible the solution needs to be done on the pyzmq side
[Thursday 02 September 2010] [08:43:16] <lestrrat> sustrik: wrt EINTR thing, what version should I be expecting it to be implemented? just want to know, cause I’d like to wait my first release of mongrel2 handler until that’s fixed.
[Thursday 02 September 2010] [08:43:35] <lestrrat> don’t want to tell my users to send a SIGKILL to stop their handlers :)
[Thursday 02 September 2010] [08:43:53] <lestrrat> (not in a hurry, just want to get an idea)
[Thursday 02 September 2010] [08:44:02] <sustrik> lestrrat: the problem is that it isn’t backward compatible
[Thursday 02 September 2010] [08:44:07] <sustrik> :|
[Thursday 02 September 2010] [08:44:08] <lestrrat> ah
[Thursday 02 September 2010] [08:44:12] <mato> EINTR again ?
[Thursday 02 September 2010] [08:44:12] <lestrrat> yeah, true
[Thursday 02 September 2010] [08:44:15] <sustrik> yes
[Thursday 02 September 2010] [08:44:34] <sustrik> it turns out that Ctrl+C problem in bindings
[Thursday 02 September 2010] [08:44:41] <sustrik> can be solved by returning EINTR
[Thursday 02 September 2010] [08:44:50] <mato> sustrik: have you verified this?
[Thursday 02 September 2010] [08:45:02] <sustrik> lestrrat did with perl binding
[Thursday 02 September 2010] [08:45:15] <mato> hmm, i think i know why it “solves” the problem
[Thursday 02 September 2010] [08:45:21] <mato> or i can guess...
[Thursday 02 September 2010] [08:45:28] <sustrik> i’m still waiting for brian to verify it with python
[Thursday 02 September 2010] [08:45:29] <lestrrat> I’m willing to accept I’m a bit off :)
[Thursday 02 September 2010] [08:45:48] <sustrik> mato: go on
[Thursday 02 September 2010] [08:46:01] <mato> i reckon what happens is returning EINTR means “the language runtime” wakes up
[Thursday 02 September 2010] [08:46:18] <sustrik> right
[Thursday 02 September 2010] [08:46:35] <mato> depending on said language runtime’s signal handling (out of our control, obviously), the runtime waking up may mean that it processes pending signals
[Thursday 02 September 2010] [08:46:50] <mato> hence the “solution”
[Thursday 02 September 2010] [08:47:06] <sustrik> well, it least the runtime has a chance to process the signal
[Thursday 02 September 2010] [08:47:16] <mato> hmm
[Thursday 02 September 2010] [08:47:20] <sustrik> right now it’s completely stuck
[Thursday 02 September 2010] [08:47:23] <mato> yeah
[Thursday 02 September 2010] [08:47:43] <mato> of course the runtime might be doing something entirely different
[Thursday 02 September 2010] [08:47:50] <mato> so it’s not a guaranteed solution
[Thursday 02 September 2010] [08:47:53] <sustrik> not our fault
[Thursday 02 September 2010] [08:47:56] <mato> just one that might help some people
[Thursday 02 September 2010] [08:48:16] <mato> hmm, except...
[Thursday 02 September 2010] [08:48:19] <sustrik> obviously we cannot give guarantees for broken runtimes
[Thursday 02 September 2010] [08:48:31] <mato> yes, but what is your proposed implementation?
[Thursday 02 September 2010] [08:48:44] <sustrik> forward EINTR to the user
[Thursday 02 September 2010] [08:48:48] <mato> yes, except...
[Thursday 02 September 2010] [08:49:08] <mato> the relationship between EINTR from a syscall and SIGINT is afaik nondeterministic
[Thursday 02 September 2010] [08:49:17] <mato> in other words, you might get an EINTR back from a blocking call
[Thursday 02 September 2010] [08:49:18] <mato> or not
[Thursday 02 September 2010] [08:49:26] <mato> you see where this is heading?
[Thursday 02 September 2010] [08:49:31] <lestrrat> should I explain how perl’s signal handling works? if you guys know what’s up, I’m not an entirely an expert either so I’ll let you guys debate the actual implementation ;)
[Thursday 02 September 2010] [08:49:43] <mato> lestrrat: sure, go for it?
[Thursday 02 September 2010] [08:49:45] <mato> .
[Thursday 02 September 2010] [08:50:56] <lestrrat> well, the important part wrt to all of this is that, when we go down to the C world, Perl’s signal handling is effectively silenced.
[Thursday 02 September 2010] [08:51:12] <lestrrat> (C world, as in binding)
[Thursday 02 September 2010] [08:51:42] <lestrrat> when the binding method is called, things are completely left up to the C/C++ routine, so you can do whatever with the signals
[Thursday 02 September 2010] [08:51:59] <lestrrat> and when zmq’s recv() is being called, perl has absolutely no say there
[Thursday 02 September 2010] [08:52:33] <lestrrat> so from perl’s perspective, all it needs is for recv() to return, and let it know from a return value or errno that there was a signal sent somewhere along the line
[Thursday 02 September 2010] [08:53:06] <lestrrat> then (if need be) the Perl binding can tell the perl interpreter that there was a signal, and let perl’s sig handlers run
[Thursday 02 September 2010] [08:53:24] <lestrrat> at least that’s how I understand it to work.
[Thursday 02 September 2010] [08:53:42] <sustrik> right
[Thursday 02 September 2010] [08:54:03] <mato> ok
[Thursday 02 September 2010] [08:54:18] <mato> i’ve not looked into the perl signal handling
[Thursday 02 September 2010] [08:54:18] <sustrik> what i think mato is saying is that OS does not have to interrupt a blocking call (at C level) when SIGINT happens
[Thursday 02 September 2010] [08:54:24] <mato> yes
[Thursday 02 September 2010] [08:54:32] <lestrrat> right
[Thursday 02 September 2010] [08:54:34] <sustrik> how come that Ctrl+C works then
[Thursday 02 September 2010] [08:54:35] <sustrik> ?
[Thursday 02 September 2010] [08:54:42] <sustrik> for C programs?
[Thursday 02 September 2010] [08:55:01] <sustrik> if main thread was stuck in a blocking call
[Thursday 02 September 2010] [08:55:02] <mato> ah, the thing is multiple threads are involved
[Thursday 02 September 2010] [08:55:08] <mato> this is what makes it tricky
[Thursday 02 September 2010] [08:55:25] <sustrik> but how does it work in C world?
[Thursday 02 September 2010] [08:55:27] <mato> ok, in the simple case
[Thursday 02 September 2010] [08:55:30] <mato> (1 thread)
[Thursday 02 September 2010] [08:55:40] <mato> the OS will invoke the default signal handler
[Thursday 02 September 2010] [08:55:46] <mato> which happens to be “exit the program”
[Thursday 02 September 2010] [08:56:10] <mato> the problem is, if you’re handling signals
[Thursday 02 September 2010] [08:56:12] <sustrik> ah, so no SIGINT
[Thursday 02 September 2010] [08:56:16] <sustrik> just hard exit
[Thursday 02 September 2010] [08:56:18] <mato> yes, you get SIGINT
[Thursday 02 September 2010] [08:56:23] <mato> it’s hard to explain...
[Thursday 02 September 2010] [08:56:32] <sustrik> try
[Thursday 02 September 2010] [08:56:46] * mato needs a whiteboard :)
[Thursday 02 September 2010] [08:56:52] <lestrrat> :)
[Thursday 02 September 2010] [08:56:54] <sustrik> ascii graphics!
[Thursday 02 September 2010] [08:57:12] <mikko> sustrik: it’s possible that i might be in Amsterdam whole of next week
[Thursday 02 September 2010] [08:57:22] <mikko> they are still trying to figure out schedules
[Thursday 02 September 2010] [08:57:28] <sustrik> aha
[Thursday 02 September 2010] [08:57:36] <sustrik> ok, we’ll see
[Thursday 02 September 2010] [08:57:37] <mato> ok, so, case 0: 1 thread, no signal handler: result: on ^C default “exit the program” handler is called by the OS
[Thursday 02 September 2010] [08:58:38] <mato> case 1: 1 thread, and a signal handler in that thread: result: on ^C, threads SIGINT handler is called and when that returns the blocking call it interrupted returns EINTR
[Thursday 02 September 2010] [08:58:50] <mato> it is now up to the program to resolve that EINTR
[Thursday 02 September 2010] [08:58:59] <mato> if it did not exit in the signal handler already, of course
[Thursday 02 September 2010] [08:59:15] <sustrik> ack
[Thursday 02 September 2010] [08:59:27] <mato> now, here’s what i imagine most language runtimes do in a signal handler
[Thursday 02 September 2010] [08:59:45] <mato> and in fact most apps with old-style (no separate thread with sigwait()) signal handlers
[Thursday 02 September 2010] [08:59:51] <mato> they set some flag
[Thursday 02 September 2010] [08:59:54] <mato> that’s all :)
[Thursday 02 September 2010] [09:00:14] <mato> hence, when the call returns EINTR, the app/Perl/whatever checks it’s “was i interrupted” flag
[Thursday 02 September 2010] [09:00:27] <mato> and if it was, deals with that in it’s normal control flow
[Thursday 02 September 2010] [09:00:36] <lestrrat> right
[Thursday 02 September 2010] [09:00:44] <sustrik> ok
[Thursday 02 September 2010] [09:00:46] <mato> now, enter 0mq
[Thursday 02 September 2010] [09:00:54] <mato> the i/o threads ignore all signals
[Thursday 02 September 2010] [09:00:56] * sustrik is thrilled
[Thursday 02 September 2010] [09:01:05] <sustrik> right
[Thursday 02 September 2010] [09:01:06] <mato> they may or may not get an EINTR if the process gets a signal
[Thursday 02 September 2010] [09:01:10] <mato> that is entirely OS-dependent
[Thursday 02 September 2010] [09:01:44] <mato> assuming for a moment that they did, the naive implementation would just return that EINTR, which would bubble up to the app thread logic and something useful might happen
[Thursday 02 September 2010] [09:01:53] <mato> however, that is a bad assumption
[Thursday 02 September 2010] [09:02:19] <mato> so, what to do?
[Thursday 02 September 2010] [09:02:20] <sustrik> wait a sec
[Thursday 02 September 2010] [09:02:28] * mato waiting
[Thursday 02 September 2010] [09:02:31] <sustrik> are you saying that I/O threadsmay get the signal;?
[Thursday 02 September 2010] [09:02:45] <mato> I/O threads will never “get” the signal
[Thursday 02 September 2010] [09:02:49] <sustrik> ok
[Thursday 02 September 2010] [09:03:03] <mato> but depending on your OS, they may get an EINTR back from any random syscall
[Thursday 02 September 2010] [09:03:09] <sustrik> ack
[Thursday 02 September 2010] [09:03:14] <mato> well, not any random, but anything that does something complex
[Thursday 02 September 2010] [09:03:21] <mato> i.e. not getpid() but send() is a candidate
[Thursday 02 September 2010] [09:03:24] <sustrik> ok, but that’s beside the point
[Thursday 02 September 2010] [09:03:31] <mato> it’s not
[Thursday 02 September 2010] [09:03:31] <sustrik> beause we are interested in app threads
[Thursday 02 September 2010] [09:03:35] <sustrik> not i/o threads
[Thursday 02 September 2010] [09:03:53] <sustrik> whether i/o thread gets sigint or not, we don’t care
[Thursday 02 September 2010] [09:04:19] <mato> yes we do, because if you were to be 100% compatible you’d have to “emulate” the behaviour i described in case 1
[Thursday 02 September 2010] [09:04:32] <mato> s/compatible/nice to broken apps/interpreters/
[Thursday 02 September 2010] [09:04:46] <sustrik> wait
[Thursday 02 September 2010] [09:04:54] <sustrik> i/o thread just loops
[Thursday 02 September 2010] [09:04:55] <lestrrat> isn’t thre recv() being called on the app thread?
[Thursday 02 September 2010] [09:05:08] <sustrik> so if it gets EINTR it can just ignore it
[Thursday 02 September 2010] [09:05:25] <mato> hmm, yes
[Thursday 02 September 2010] [09:05:30] <sustrik> lestrrat: that’s my point
[Thursday 02 September 2010] [09:05:49] <mato> ok, right, so you want the app thread side of the API to pass EINTR
[Thursday 02 September 2010] [09:05:54] <sustrik> ack
[Thursday 02 September 2010] [09:05:56] <lestrrat> right
[Thursday 02 September 2010] [09:06:18] <mato> hmm
[Thursday 02 September 2010] [09:06:33] <mato> ok, one way to do it w/o breaking compatibility
[Thursday 02 September 2010] [09:06:36] <mato> with existing code
[Thursday 02 September 2010] [09:06:49] <mato> is a context option
[Thursday 02 September 2010] [09:06:57] <mato> “ZMQ_INTERRUPTIBLE”
[Thursday 02 September 2010] [09:07:09] <sustrik> ok, so it would work?
[Thursday 02 September 2010] [09:07:11] <mato> meaning “API calls will return EINTR if interrupted by a signal”
[Thursday 02 September 2010] [09:07:19] <mato> i think so
[Thursday 02 September 2010] [09:07:24] <sustrik> woohoo
[Thursday 02 September 2010] [09:07:30] <mato> note that there is no change on the i/o thread side
[Thursday 02 September 2010] [09:07:35] <lestrrat> coolness
[Thursday 02 September 2010] [09:07:52] <sustrik> or a compile time option
[Thursday 02 September 2010] [09:07:59] <mato> no, compile time is bad
[Thursday 02 September 2010] [09:08:04] <mato> libzmq.so may be shared
[Thursday 02 September 2010] [09:08:06] <lestrrat> I’d vote for ZMQ_INTERRUPTABLE
[Thursday 02 September 2010] [09:08:17] <sustrik> there are no context-wide options
[Thursday 02 September 2010] [09:08:26] <mato> well, they’ll have to be added then :)
[Thursday 02 September 2010] [09:08:40] <sustrik> but this is a hack
[Thursday 02 September 2010] [09:08:44] <mato> ?
[Thursday 02 September 2010] [09:08:57] <sustrik> the right solution is tu return EINTR
[Thursday 02 September 2010] [09:09:22] <sustrik> we need the option only to stay backward compatible with the original lousy solution
[Thursday 02 September 2010] [09:09:39] <mato> hmm hmm
[Thursday 02 September 2010] [09:09:49] <sustrik> i don’t like changing API because of a hack
[Thursday 02 September 2010] [09:10:01] <mato> sustrik: just a minute, i’m still thinking
[Thursday 02 September 2010] [09:10:06] <lestrrat> I was wondering from the beginning, but does it have to “return” EINTR? is it not enough to keep errno = EINTR ?
[Thursday 02 September 2010] [09:10:24] <sustrik> errno = EINTR
[Thursday 02 September 2010] [09:10:26] <mato> lestrrat: yes, we mean return (-1) with errno = EINTR
[Thursday 02 September 2010] [09:10:33] <lestrrat> ah
[Thursday 02 September 2010] [09:10:39] <sustrik> but still it breaks the API specification
[Thursday 02 September 2010] [09:10:54] <sustrik> EINTR is not described as valid error from recv()
[Thursday 02 September 2010] [09:11:04] <mato> recv() or zmq_recv()?
[Thursday 02 September 2010] [09:11:19] <sustrik> zmq_recv(), sorry
[Thursday 02 September 2010] [09:11:30] <mato> so we’ll have to change that, i think
[Thursday 02 September 2010] [09:11:38] <sustrik> people will hate us
[Thursday 02 September 2010] [09:11:52] <mato> then add ZMQ_INTERRUPTIBLE
[Thursday 02 September 2010] [09:11:57] <sustrik> :)
[Thursday 02 September 2010] [09:12:06] <lestrrat> hey, I’m willing to accept a zmq_recv2() ;P
[Thursday 02 September 2010] [09:12:07] <mato> the thing is, the model of never getting EINTR is actually right if you do your handling properly
[Thursday 02 September 2010] [09:12:28] <mato> kind of
[Thursday 02 September 2010] [09:12:33] <sustrik> shrug
[Thursday 02 September 2010] [09:12:48] <lestrrat> does that mean the “correct” way is for me to install a sighandler in my binding?
[Thursday 02 September 2010] [09:13:08] <mato> lestrrat: that probably won’t help because of the way the interpreter is architected
[Thursday 02 September 2010] [09:13:22] <mato> lestrrat: but yes, that would be the correct way
[Thursday 02 September 2010] [09:13:29] <mato> in an ideal world :)
[Thursday 02 September 2010] [09:13:33] <sustrik> in broader sense i would say: let’s keep with POSIX API
[Thursday 02 September 2010] [09:13:45] <sustrik> trying to outsmart it just causes problems
[Thursday 02 September 2010] [09:13:45] <lestrrat> oh yeah, recv is a currently a loop, ain’t it...
[Thursday 02 September 2010] [09:13:58] <mato> sustrik: well, what this requires then is...
[Thursday 02 September 2010] [09:14:10] <mato> sustrik: a nice thorough explanation by email
[Thursday 02 September 2010] [09:14:16] <mato> sustrik: combined with “sorry, we messed up”
[Thursday 02 September 2010] [09:14:23] <sustrik> we did
[Thursday 02 September 2010] [09:14:27] <mato> sustrik: and it’ll just go into 2.1
[Thursday 02 September 2010] [09:14:37] <mato> eventually, not immediately
[Thursday 02 September 2010] [09:14:58] <mato> after actual verification with at least say Perl, Python, Ruby that it does solve the problem
[Thursday 02 September 2010] [09:15:18] <sustrik> actually, it was brian granger who asked for backward compatibility guarantees :)
[Thursday 02 September 2010] [09:15:27] <lestrrat> would it be acceptable to add a different, aptly named function?
[Thursday 02 September 2010] [09:15:41] <lestrrat> zmq_recv_intr() or whatever.
[Thursday 02 September 2010] [09:15:41] <mato> lestrrat: no, because it involves ALL API calls that can block
[Thursday 02 September 2010] [09:15:53] <lestrrat> hmm
[Thursday 02 September 2010] [09:16:07] <mato> it is actually a mistake on our part, but i didn’t realise until today why it was a mistake
[Thursday 02 September 2010] [09:16:32] <mato> well, we could be anal and say “make your signal handling work like this”, but that’s impossible in the real world
[Thursday 02 September 2010] [09:17:02] <sustrik> ok, let me write an email describing the problem
[Thursday 02 September 2010] [09:17:11] <mato> do you understand it well enough?
[Thursday 02 September 2010] [09:17:21] <mato> this email must not be compressed :-)
[Thursday 02 September 2010] [09:17:24] <sustrik> and asking whether breaking backward compatibility is acceptable in this case
[Thursday 02 September 2010] [09:17:25] <sustrik> no
[Thursday 02 September 2010] [09:17:29] <sustrik> you can write it
[Thursday 02 September 2010] [09:17:44] <sustrik> but it involves saying “i am an idiot, sorry”
[Thursday 02 September 2010] [09:17:49] <mato> i’d like to verify that it actually helps
[Thursday 02 September 2010] [09:18:00] <mato> yes, true, you were the one that defined the behaviour
[Thursday 02 September 2010] [09:18:14] <mato> ok, look, i’ll write the text describing the problem
[Thursday 02 September 2010] [09:18:23] <mato> you can send it out, adding “I’m an idiot”
[Thursday 02 September 2010] [09:18:27] <mato> ok? :)
[Thursday 02 September 2010] [09:18:27] <sustrik> :)
[Thursday 02 September 2010] [09:18:41] <sustrik> ok, let’s first check whether it helps
[Thursday 02 September 2010] [09:18:44] <mato> but before you publicy denounce yourself, it would be nice to check first
[Thursday 02 September 2010] [09:18:46] <mato> precisely
[Thursday 02 September 2010] [09:18:58] <mato> sustrik: i would suggest being very pendantic about this
[Thursday 02 September 2010] [09:19:00] <sustrik> if it turns out that it does not i don’t have to call myself an idiotr
[Thursday 02 September 2010] [09:19:12] <mato> i.e. make three test cases (Perl, Python, Ruby)
[Thursday 02 September 2010] [09:19:17] <sustrik> ok, i can create a topic branch
[Thursday 02 September 2010] [09:19:19] <mato> ensure they hang currently
[Thursday 02 September 2010] [09:19:21] <sustrik> fix it there
[Thursday 02 September 2010] [09:19:24] <mato> i.e. ^C doesn’t work
[Thursday 02 September 2010] [09:19:31] <mato> then make your change
[Thursday 02 September 2010] [09:19:36] <mato> and ensure that all works as expected
[Thursday 02 September 2010] [09:19:55] <mato> sustrik: yes, topic branch, involves doc changes and so on
[Thursday 02 September 2010] [09:19:57] <mato> good idea
[Thursday 02 September 2010] [09:20:11] <sustrik> ok, let me do it
[Thursday 02 September 2010] [09:20:14] <lestrrat> let me know when I can test it :)
[Thursday 02 September 2010] [09:20:19] <mato> sustrik: i will find the magic command for you to email the topic branch patch set around
[Thursday 02 September 2010] [09:20:22] <sustrik> lestrrat: i’ll ping you
[Thursday 02 September 2010] [09:20:30] <sustrik> i’ll ping brian as well
[Thursday 02 September 2010] [09:20:32] <mato> sustrik: so that you can give it to e.g. lestrrat
[Thursday 02 September 2010] [09:20:33] <mato> ja
[Thursday 02 September 2010] [09:20:46] <sustrik> he’s willing to test it, we’ve discussed it in the morning
[Thursday 02 September 2010] [09:20:54] <mato> just email the patches around privately if you don’t want to call yourself a potential idiot just yet :)
[Thursday 02 September 2010] [09:20:56] <sustrik> not sure about ruby
[Thursday 02 September 2010] [09:20:58] <mato> (in public) :)
[Thursday 02 September 2010] [09:21:03] <sustrik> cremes: are you here?
[Thursday 02 September 2010] [09:21:53] <lestrrat> +1 for branch (just easier to pull ;)
[Thursday 02 September 2010] [09:22:13] <sustrik> lestrrat: what’s you email
[Thursday 02 September 2010] [09:22:18] <sustrik> how should i ping you?
[Thursday 02 September 2010] [09:22:24] <sustrik> thx
[Thursday 02 September 2010] [09:22:30] <lestrrat> lestrrat @ github, lestrrat @ twitter
[Thursday 02 September 2010] [09:22:34] <sustrik> ok
[Thursday 02 September 2010] [09:22:34] <mato> lestrrat: shhh... i’m slowly teaching sustrik git
[Thursday 02 September 2010] [09:22:42] <lestrrat> lol
[Thursday 02 September 2010] [09:22:52] <mato> start with local branches
[Thursday 02 September 2010] [09:22:53] <mato> :)
[Thursday 02 September 2010] [09:24:07] <sustrik> ok
[Thursday 02 September 2010] [09:27:28] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [09:29:01] <mato> ok, i have to concentrate on something else for a bit
[Thursday 02 September 2010] [09:29:12] <mato> ping me if you need me, bbl
[Thursday 02 September 2010] [09:31:39] <sustrik> cya
[Thursday 02 September 2010] [09:53:57] <cremes> sustrik: just got here; what do you need?
[Thursday 02 September 2010] [09:54:25] <sustrik> are you seeing problem with Ctrl+C in Ruby?
[Thursday 02 September 2010] [09:54:46] <sustrik> i mean, application not responding to SIGINT?
[Thursday 02 September 2010] [09:55:10] <sustrik> when stuck in 0mq blocking call?
[Thursday 02 September 2010] [09:55:13] <cremes> yes
[Thursday 02 September 2010] [09:55:24] <sustrik> aha, good
[Thursday 02 September 2010] [09:55:31] <sustrik> we think we’ve found a solution
[Thursday 02 September 2010] [09:55:32] <cremes> i don’t think the ruby signal handler runs when external C code is executing
[Thursday 02 September 2010] [09:55:39] <cremes> really? that’s good
[Thursday 02 September 2010] [09:55:54] <sustrik> would you be willing to test it once i have a fix?
[Thursday 02 September 2010] [09:56:33] <sustrik> the idea is that the blocking calls would return EINTR in case of Ctrl+C
[Thursday 02 September 2010] [09:56:57] <sustrik> then the binding can take care or what happens next
[Thursday 02 September 2010] [09:57:11] <cremes> sure
[Thursday 02 September 2010] [09:57:20] <sustrik> great, i’ll ping you once i have it
[Thursday 02 September 2010] [09:57:25] <cremes> ok
[Thursday 02 September 2010] [10:05:49] <mrm2m> ignore those random and threading they are not used.
[Thursday 02 September 2010] [10:06:41] <mrm2m> the “thread started” shows up, but nothing is sent to the corresponding server.
[Thursday 02 September 2010] [10:09:48] <sustrik> mrm2m: the application exits after sending the message?
[Thursday 02 September 2010] [10:10:00] <mrm2m> right
[Thursday 02 September 2010] [10:10:30] <sustrik> in 2.0.8 all the unsent data are discarded when socket is closed
[Thursday 02 September 2010] [10:10:54] <mrm2m> Uh - ok
[Thursday 02 September 2010] [10:11:19] <sustrik> the semantics is changed in 2.1
[Thursday 02 September 2010] [10:11:39] <sustrik> it’s: block context termination while all data are sent
[Thursday 02 September 2010] [10:12:23] <guido_g> so you can’t exit if something is broken?
[Thursday 02 September 2010] [10:12:44] <sustrik> it’s annoying, i know
[Thursday 02 September 2010] [10:13:02] <sustrik> what i want to do is to add SO_LINGER socket option
[Thursday 02 September 2010] [10:13:12] <lestrrat> would be nice if it was configurable
[Thursday 02 September 2010] [10:13:18] <guido_g> ack
[Thursday 02 September 2010] [10:13:18] <sustrik> with same semantics as with POSIX sockets
[Thursday 02 September 2010] [10:13:42] <sustrik> that should do imo
[Thursday 02 September 2010] [10:16:23] Quit bgranger has left this server (Quit: bgranger).
[Thursday 02 September 2010] [11:07:11] Quit poswald has left this server (Quit: poswald).
[Thursday 02 September 2010] [11:10:07] Quit mrm2m has left this server (Quit: Leaving.).
[Thursday 02 September 2010] [11:11:30] Quit poswald has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [11:25:43] <gavinstark> I am following the PUB/SUB example in the ‘guide’ but I’m wondering how I might setup a way for there to be multiple publishers that each subscriber can receive from. Or do I need independent bind/connects for that?
[Thursday 02 September 2010] [11:27:54] <mikko> gavinstark: the latter
[Thursday 02 September 2010] [11:34:29] <cremes> gavinstark: you also need a forwarder device to aggregate all of the publisher’s messages
[Thursday 02 September 2010] [11:35:01] <cremes> (not strictly true, but i think it’s the cleanest way to set things up)
[Thursday 02 September 2010] [11:42:47] <gavinstark> Do I have to pre-list all the publishers in the forwarder config? It seems like I’d have to have multiple <in> with unique values for each publisher that might appear? Or am I missing something?
[Thursday 02 September 2010] [11:55:00] <sustrik> gavinstark: bind your forwarder devices
[Thursday 02 September 2010] [11:55:12] <sustrik> connect the publishers and subscribers
[Thursday 02 September 2010] [11:57:42] <gavinstark> sustrik, what if I do not know the qty of publishers before hand? Won’t each publisher have to bind uniquely? (
tcp://....:5555,
tcp://....:5556, etc.?)
[Thursday 02 September 2010] [11:59:07] <sustrik> pubishers should _connect_ to the forwared
[Thursday 02 September 2010] [11:59:11] <sustrik> forwarder*
[Thursday 02 September 2010] [12:06:44] <gavinstark> surstrik, ah, ok. I just tried that, having the publisher zmq_connect instead of bind, still not quite working. Here is what I did:
[Thursday 02 September 2010] [12:08:10] <gavinstark> ah, I think I see, I was supposed to “bind” on the “in” entry?
[Thursday 02 September 2010] [12:12:45] <sustrik> gavinstark: forwarder should bind both in and out
[Thursday 02 September 2010] [12:12:47] <zedas> lestrrat: re: mongrel2 handlers blocking, i fixed it by making the zeromq IO threads be 1. I’ve been running it like that for weeks without any problems, so you should be fine.
[Thursday 02 September 2010] [12:13:12] <gavinstark> sustrik: Thanks, working perfectly now.
[Thursday 02 September 2010] [12:13:15] <sustrik> zedas: this is a different issue
[Thursday 02 September 2010] [12:13:50] <sustrik> annoying interactions between language runime, OS signals and 0MQ async architecture
[Thursday 02 September 2010] [12:15:02] <zedas> sustrik: ah. you got a link for me about it?
[Thursday 02 September 2010] [12:15:04] <mato> that’s a very polite way of putting it :-)
[Thursday 02 September 2010] [12:15:54] <sustrik> mato: are you able to summarise it for zed?
[Thursday 02 September 2010] [12:16:00] <mato> yeah
[Thursday 02 September 2010] [12:16:04] * sustrik feels incompetent here
[Thursday 02 September 2010] [12:16:08] <mato> signal handling is fucked :-)
[Thursday 02 September 2010] [12:16:11] <mato> end of summary
[Thursday 02 September 2010] [12:16:32] <sustrik> up to the point
[Thursday 02 September 2010] [12:17:04] <sustrik> basically, it has to do with handling Ctrl+C in interpreted languages
[Thursday 02 September 2010] [12:17:07] <mato> to elaborate on my summary, the issue is with the 0mq API not returning EINTR when API calls are interrupted by a signal in the application thread
[Thursday 02 September 2010] [12:17:09] <sustrik> i’m working on a fix now
[Thursday 02 September 2010] [12:17:23] <mato> zedas: problem is most language runtimes do delayed signal handling
[Thursday 02 September 2010] [12:17:33] <mato> zedas: i.e. handler() just sets some flag
[Thursday 02 September 2010] [12:17:39] <mato> zedas: it doesn’t actually do anything
[Thursday 02 September 2010] [12:17:46] <mato> zedas: flag gets picked up when the runtime wakes up
[Thursday 02 September 2010] [12:18:01] <mato> zedas: but since 0mq calls never return EINTR, runtime never wakes up
[Thursday 02 September 2010] [12:22:32] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Thursday 02 September 2010] [12:25:08] Quit keffo has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [12:26:22] <zedas> mato: ah yes, that’d explain it.
[Thursday 02 September 2010] [12:35:20] <ModusPwnens> hi, i’m back again. Hopefully with a question that’s not as dumb this time. Anyways, does anyone know what would cause the first two bytes of a message to get corrupted?
[Thursday 02 September 2010] [12:39:20] <bgranger> zedas: can you say more about what you have been doing to solve this. I was going to look into the EINTR stuff today.
[Thursday 02 September 2010] [12:40:19] <mato> bgranger: I think we have a solution for the EINTR stuff
[Thursday 02 September 2010] [12:40:27] <sustrik> ModusPwnens: do you have a test program?
[Thursday 02 September 2010] [12:40:46] <bgranger> mato: ?
[Thursday 02 September 2010] [12:40:48] <mato> bgranger: oh, hang on, maybe you mean zed’s issue
[Thursday 02 September 2010] [12:40:56] <bgranger> Is that not the same thing?
[Thursday 02 September 2010] [12:41:07] <mato> as opposed to the general issue which manifests itself as “^C doesn’t work in $RANDOM_LANGUAGE”
[Thursday 02 September 2010] [12:41:09] <bgranger> I missed some of the discussion so I am trying to piece it together
[Thursday 02 September 2010] [12:41:14] <mato> ah
[Thursday 02 September 2010] [12:41:24] <bgranger> Are there 2 different issues?
[Thursday 02 September 2010] [12:41:27] <mato> yeah
[Thursday 02 September 2010] [12:41:42] <mato> zedas has an issue with multiple i/o threads
[Thursday 02 September 2010] [12:41:53] <mato> which may or may not have something (different) to do with EINTR
[Thursday 02 September 2010] [12:42:00] <mato> not clear, haven’t had time to look at that
[Thursday 02 September 2010] [12:42:00] <bgranger> And signals?
[Thursday 02 September 2010] [12:42:03] <mato> signals
[Thursday 02 September 2010] [12:42:11] <mato> The problem is as I described just above.
[Thursday 02 September 2010] [12:42:18] <bgranger> What is the idea about EINTR?
[Thursday 02 September 2010] [12:42:33] <mato> When Martin Sustrik made the original API he decided not to return EINTR from blocking calls
[Thursday 02 September 2010] [12:42:39] <bgranger> I did talk to sustrik late last night (for me) and he pointed me to some code ...
[Thursday 02 September 2010] [12:42:46] <bgranger> Right he showed me that code
[Thursday 02 September 2010] [12:43:01] <mato> Right, except that basically breaks standard signal handling
[Thursday 02 September 2010] [12:43:13] <bgranger> Today I am going to put in a print statement in that logic and try to see if it prints with a SIGINT in the Python bindings.
[Thursday 02 September 2010] [12:43:42] <bgranger> It is still not clear if this will work with the Python bindings, but we will see.
[Thursday 02 September 2010] [12:43:43] <mato> To recap, imagine the simplest case with a C program blocking on some syscall, while at the same time handling say SIGINT.
[Thursday 02 September 2010] [12:43:44] <bgranger> I think it may
[Thursday 02 September 2010] [12:43:49] <ModusPwnens> sustrik: Yeah, it’s happening in my code. I’m not entirely sure why. It seems like it happens when it is sent, as the sending side can properly decode the message. However, the receving side cannot because the first two bytes get corrupted for some reason. I can paste my code to a pastebin if you like.
[Thursday 02 September 2010] [12:44:22] <mato> bgranger: Now, if “handling” SIGINT in this programs case means it just prints “Interrupted!” and exits, then fine.
[Thursday 02 September 2010] [12:44:33] <mato> bgranger: That will work even with the current situation in 0MQ.
[Thursday 02 September 2010] [12:44:53] <bgranger> Not in the python bindings...
[Thursday 02 September 2010] [12:45:10] <mato> bgranger: But, if it instead means that the program just sets some random flag, and then expects to process that flag “later”, it won’t work.
[Thursday 02 September 2010] [12:45:19] <bgranger> Right
[Thursday 02 September 2010] [12:45:27] <mato> bgranger: Which is precisely the Python/Perl (at least) case
[Thursday 02 September 2010] [12:45:32] <bgranger> Which is what Python does...yep
[Thursday 02 September 2010] [12:45:55] <sustrik> ModusPwnens: try it
[Thursday 02 September 2010] [12:45:56] <mato> bgranger: So, the only real solution is that 0MQ API calls return EINTR if they get EINTR back from a blocking system call.
[Thursday 02 September 2010] [12:46:12] <bgranger> Will that happen regardless of what signal handlers have been installed?
[Thursday 02 September 2010] [12:46:19] <mato> Yup
[Thursday 02 September 2010] [12:46:20] <ModusPwnens> sustrik: do you want the entire code or just the functions in question?
[Thursday 02 September 2010] [12:46:36] <mato> bgranger: It’s what the OS does.
[Thursday 02 September 2010] [12:46:40] <bgranger> That would definittely solve our problems then! e would be very happy about that.
[Thursday 02 September 2010] [12:46:49] <sustrik> well, i would like a simple example
[Thursday 02 September 2010] [12:46:57] <sustrik> showing the problem
[Thursday 02 September 2010] [12:47:05] <mato> bgranger: If you do e.g. poll () in C, handle a signal in the same thread, then that poll () will return EINTR
[Thursday 02 September 2010] [12:47:19] <mato> bgranger: by “handle a signal” I mean the “set a flag case”
[Thursday 02 September 2010] [12:47:23] <sustrik> bgranger: i’ll fix it and let you know
[Thursday 02 September 2010] [12:47:24] <bgranger> mato: right OK
[Thursday 02 September 2010] [12:47:39] <sustrik> you can test it with python, others will test with perl and ruby
[Thursday 02 September 2010] [12:47:43] <mato> bgranger: By the way, while you’re here, have you had any more Python/GIL issues?
[Thursday 02 September 2010] [12:47:49] <bgranger> sustrik: can you fix it in the 2.0.8 branch. We are not using trunk yet
[Thursday 02 September 2010] [12:48:09] <sustrik> backwards compatibility :|
[Thursday 02 September 2010] [12:48:11] <bgranger> mato: No we have solved those. It was super sutble to get non-copy send/recv working with the GIL though.
[Thursday 02 September 2010] [12:48:20] <bgranger> mato: But it is working well.
[Thursday 02 September 2010] [12:48:24] <mato> bgranger: I have a really interesting case which looks like one of those...
[Thursday 02 September 2010] [12:48:44] <mato> bgranger: Will send to the mailing list, tomorrow.
[Thursday 02 September 2010] [12:49:06] <ModusPwnens> Hmm.well i don’t really have an example..it’s just happening in the code I have written, so i could give you that if you wanted to see it. It started happening after i used google protobufs to encode into a byte array instead of a string.
[Thursday 02 September 2010] [12:49:11] <bgranger> mato: Ok, I will watch for it. The challenge was getting the ref counts of zmq messages synch’d with those of Python.
[Thursday 02 September 2010] [12:49:30] <mato> bgranger: But the short story is, I see gc trying to close() a socket, while at the same time zmq_free_fn() in a different thread is trying to acquire the GIL in order to decrease the message refcount
[Thursday 02 September 2010] [12:49:39] <mato> bgranger: and the result is deadlock
[Thursday 02 September 2010] [12:50:01] <bgranger> Is this in trunk where sockets can move threads?
[Thursday 02 September 2010] [12:50:08] <mato> bgranger: nope, 2.0.8
[Thursday 02 September 2010] [12:50:22] <bgranger> What language?
[Thursday 02 September 2010] [12:50:27] <mato> Python...
[Thursday 02 September 2010] [12:50:35] <bgranger> What version of pyzmq?
[Thursday 02 September 2010] [12:50:55] <mato> latest-ish, let me check
[Thursday 02 September 2010] [12:51:07] <bgranger> Since Saturday?
[Thursday 02 September 2010] [12:51:10] <mato> ah, no
[Thursday 02 September 2010] [12:51:16] <mato> 2.0.7 pyzmq actually
[Thursday 02 September 2010] [12:51:22] <mato> with 2.0.8 zmq
[Thursday 02 September 2010] [12:51:23] <bgranger> I did a bunch of work then. I release a 2.0.7 stable release and master is now 2.0.8 cmpatible.
[Thursday 02 September 2010] [12:51:58] <bgranger> But, I believe that what you are saying it possible. The 1 problem we have is that the zmq_free_fn does have to acquire the GIL. If that can’t happen, you have trouble.
[Thursday 02 September 2010] [12:52:01] <jonrafkind> has anyone used zmq in a real-time game? mostly I just need low latency
[Thursday 02 September 2010] [12:52:13] <mato> bgranger: that’s precisely what I’m seeing
[Thursday 02 September 2010] [12:52:13] <bgranger> There is nothing we can do to get around this.
[Thursday 02 September 2010] [12:52:32] <bgranger> Can you just hold onto the socket ref to prevent gc?
[Thursday 02 September 2010] [12:52:44] <mato> Yes, but I have transient sockets in this application
[Thursday 02 September 2010] [12:52:49] <mato> So tons of fds get leaked
[Thursday 02 September 2010] [12:52:57] <bgranger> Hmmm, that might be tough
[Thursday 02 September 2010] [12:53:06] <bgranger> What do you mean by that?
[Thursday 02 September 2010] [12:53:22] <mato> Well, it’s precisely my workaround (holding onto the socket refs)
[Thursday 02 September 2010] [12:53:29] <bgranger> Ahh, OK.
[Thursday 02 September 2010] [12:53:45] <mato> But that means the underlying fds hang around, so eventually you’ll run out.
[Thursday 02 September 2010] [12:53:46] <bgranger> Is the socket that is being gc’s the one that is sending the msg though?
[Thursday 02 September 2010] [12:53:53] <bgranger> right, you don’t want that
[Thursday 02 September 2010] [12:54:10] <mato> That’s hard to tell at the moment, but at least you’ve confirmed that this can happen.
[Thursday 02 September 2010] [12:54:18] <mato> So the backtraces I see make sense.
[Thursday 02 September 2010] [12:54:22] <mato> I’ll write it up tomorrow.
[Thursday 02 September 2010] [12:54:39] <sustrik> ModusPwnens: if you want me to look at it, strip it down to the simplest possible example that reproduces the bahviour
[Thursday 02 September 2010] [12:54:53] <bgranger> I can do the following. When a socket send a msg, it can add itself to a list of sockets that the Message hold on to. That way the message can prevent the gc, but when the msg goes away, the socket will as well.
[Thursday 02 September 2010] [12:55:09] <sustrik> ModusPwnens: aren’t you overwriting the buffer you’ve sent to 0MQ?
[Thursday 02 September 2010] [12:55:30] <bgranger> but that might keep socket around longer than you want if you have a message that love a long time.
[Thursday 02 September 2010] [12:55:46] <mato> bgranger: That might be a solution, yes.
[Thursday 02 September 2010] [12:55:54] <mato> bgranger: Anyhow, gc is not supposed to be instant, no?
[Thursday 02 September 2010] [12:56:02] <bgranger> Depends
[Thursday 02 September 2010] [12:56:13] <bgranger> if there are cycles or not.
[Thursday 02 September 2010] [12:56:18] <mato> bgranger: So if the socket hangs around for a bit... does it matter too much? As long as it goes away eventually.
[Thursday 02 September 2010] [12:56:30] <sustrik> jonrafkind: i think there are couple of game devs here, rbraley for example
[Thursday 02 September 2010] [12:56:35] <bgranger> Depends on what “a bit” means
[Thursday 02 September 2010] [12:56:48] <mato> True.
[Thursday 02 September 2010] [12:56:53] <ModusPwnens> Sustrik: Well it’s in a loop and I close the message at the end of the loop, initializing it again at the beginning
[Thursday 02 September 2010] [12:57:07] <mato> bgranger: Let me sleep on it, and write up, this is useful to have on the list
[Thursday 02 September 2010] [12:57:12] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [12:57:20] <mato> bgranger: since other people dealing with GC-based languages may run into similar problems.
[Thursday 02 September 2010] [12:57:22] <bgranger> mato: Great
[Thursday 02 September 2010] [12:57:26] <bgranger> Aboslutely
[Thursday 02 September 2010] [12:57:33] <ModusPwnens> and the data that I am initializing it with is a char * which i free at the end of each iteration too
[Thursday 02 September 2010] [12:57:55] <sustrik> show me the sending code
[Thursday 02 September 2010] [12:58:10] <mato> bgranger: Thanks for your help
[Thursday 02 September 2010] [12:58:25] * mato is off to dinner, see you all later
[Thursday 02 September 2010] [12:58:56] <bgranger> later
[Thursday 02 September 2010] [13:00:11] <ModusPwnens> there’s a lot of debugging stuff in there so i’m sorry that it is messy
[Thursday 02 September 2010] [13:00:11] <sustrik> ModusPwnens: when using zmq_msg_init_data you are passing ownership of the buffer to 0MQ
[Thursday 02 September 2010] [13:00:23] <sustrik> so you have to give it a free function
[Thursday 02 September 2010] [13:00:31] <sustrik> and don’t touch the buffer afterwards
[Thursday 02 September 2010] [13:01:12] <sustrik> if you don’t need zero-copy
[Thursday 02 September 2010] [13:01:22] <sustrik> just init the message using zmq_msg_init_size
[Thursday 02 September 2010] [13:01:29] <sustrik> and copt the data into i
[Thursday 02 September 2010] [13:01:30] <sustrik> it
[Thursday 02 September 2010] [13:01:59] <ModusPwnens> So just don’t use init_data at all?
[Thursday 02 September 2010] [13:02:50] <sustrik> do you need zero-copy?
[Thursday 02 September 2010] [13:03:05] <ModusPwnens> I’m not really sure what that is, so I don’t think so.
[Thursday 02 September 2010] [13:03:14] <sustrik> than don’t use it :)
[Thursday 02 September 2010] [13:03:30] <ModusPwnens> Ok! So just use size to initialize it and then memcpy into it?
[Thursday 02 September 2010] [13:03:36] <sustrik> exactly
[Thursday 02 September 2010] [13:03:42] <ModusPwnens> okie doke. Thanks! I will try that!
[Thursday 02 September 2010] [13:03:48] <ModusPwnens> Sorry for the constant questions :S
[Thursday 02 September 2010] [13:04:29] <sustrik> np
[Thursday 02 September 2010] [13:06:09] <jonrafkind> oh i just realied, imatrix is the same company that made SFL. I use that in my project :p
[Thursday 02 September 2010] [13:33:45] Quit gebi has left this server (Ping timeout: 272 seconds).
[Thursday 02 September 2010] [14:02:40] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [14:19:52] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [14:30:16] Join gebi has joined this channel (
~gebi@84.119.80.149).
[Thursday 02 September 2010] [15:22:39] Quit plq has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:43:34] Quit
snerd_ has left this server (Changing host).
[Thursday 02 September 2010] [15:47:33] Quit snerd has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:47:35] Quit impl has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:57:13] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [16:27:52] Quit xla has left this server (Quit: leaving).
[Thursday 02 September 2010] [16:29:01] Quit ModusPwnens has left this server.
[Thursday 02 September 2010] [16:50:50] Quit gavinstark has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [16:50:52] <ModusPwnens> Hi guys, I have encountered a problem with the official benchmarking utility.
[Thursday 02 September 2010] [16:51:31] <ModusPwnens> It appears to crash if you enter in a very large number of messages, and I was w ndering if this was supposed to happen and if so, why?
[Thursday 02 September 2010] [16:58:49] Nick
impl_ is now known as impl.
[Thursday 02 September 2010] [17:16:55] <cremes> ModusPwnens: which benchmarking utility and what number did you pass to it?
[Thursday 02 September 2010] [17:17:41] <ModusPwnens> The site has changed so I don’t know where the utility is offhand anymore
[Thursday 02 September 2010] [17:18:09] <cremes> what’s the name of it?
[Thursday 02 September 2010] [17:18:31] <ModusPwnens> actually, it’s in my zeromq folder
[Thursday 02 September 2010] [17:18:35] <ModusPwnens> in the bin
[Thursday 02 September 2010] [17:18:38] <ModusPwnens> remote_thr
[Thursday 02 September 2010] [17:18:52] <cremes> oh, the local_thr/remote_thr pair?
[Thursday 02 September 2010] [17:18:55] <ModusPwnens> ya
[Thursday 02 September 2010] [17:19:17] <cremes> so what arguments did you pass it? (i recommend you pastie the output from your shell along with any displayed error)
[Thursday 02 September 2010] [17:19:32] <ModusPwnens> i passed 50 and 25000000
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> C:UsersDavid DawsonDesktopzeromq-2.0.7zeromq-2.0.7bin>remote_thr.exe tcp:/
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> Assertion failed: end_chunk->next (c:usersdavid dawsondesktopzeromq-2.0.7ze
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> romq-2.0.7srcyqueue.hpp:108)
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> This application has requested the Runtime to terminate it in an unusual way.
[Thursday 02 September 2010] [17:20:01] <ModusPwnens> Please contact the application’s support team for more information.
[Thursday 02 September 2010] [17:20:32] <cremes> did it fail only with the 25 million number or with 50 too?
[Thursday 02 September 2010] [17:20:34] <ModusPwnens> bah, sorry, the command prompt is strange
[Thursday 02 September 2010] [17:20:42] <ModusPwnens> no, it’s just the 25 million
[Thursday 02 September 2010] [17:20:50] <ModusPwnens> if i lower it it works fine
[Thursday 02 September 2010] [17:20:58] <ModusPwnens> but I was just curious if it is supposed to fail that way
[Thursday 02 September 2010] [17:21:07] <cremes> no, you may have found a bug
[Thursday 02 September 2010] [17:21:16] <cremes> but before reporting it, install 2.0.8 and try again
[Thursday 02 September 2010] [17:21:18] <ModusPwnens> ok
[Thursday 02 September 2010] [17:21:22] <cremes> no sense in reporting a bug against an old release
[Thursday 02 September 2010] [17:21:32] <ModusPwnens> true enough
[Thursday 02 September 2010] [17:22:10] <ModusPwnens> ok i have to recompile the source, hold on
[Thursday 02 September 2010] [17:22:34] <cremes> just a note on irc etiquette...
[Thursday 02 September 2010] [17:22:44] <cremes> give as much information as possible...
[Thursday 02 September 2010] [17:23:11] <cremes> don’t paste more than 2 lines directly into the channel; use a pastie srevice like pastie.org or gist.github.com for longer stuff
[Thursday 02 September 2010] [17:23:25] <cremes> tell us the name of the programs involved and the version of the library
[Thursday 02 September 2010] [17:23:43] Quit dos000 has left this server (Read error: Connection reset by peer).
[Thursday 02 September 2010] [17:23:48] <ModusPwnens> Ok. I will note that for the future.
[Thursday 02 September 2010] [17:23:56] <ModusPwnens> thanks!
[Thursday 02 September 2010] [17:23:56] <cremes> if you see a crash, it is never supposed to happen so asking if it is seems a bit silly
[Thursday 02 September 2010] [17:24:06] <cremes> np
[Thursday 02 September 2010] [17:24:09] <ModusPwnens> Well, i guess I meant to say if it was already known
[Thursday 02 September 2010] [17:24:47] <cremes> then search the issues on github; all known bugs are reported and tracked there
[Thursday 02 September 2010] [17:25:08] <ModusPwnens> Hmm, ok. I didn’t know about that..
[Thursday 02 September 2010] [17:25:43] <cremes> and now you know! ;)
[Thursday 02 September 2010] [17:25:54] <ModusPwnens> that would be under the issues section?
[Thursday 02 September 2010] [17:26:12] <cremes> correct
[Thursday 02 September 2010] [17:26:34] <ModusPwnens> Okay. Sorry about that..
[Thursday 02 September 2010] [17:27:51] <cremes> we all had to learn it at some point, so don’t worry about it
[Thursday 02 September 2010] [17:28:13] Quit dos000 has left this server (Ping timeout: 258 seconds).
[Thursday 02 September 2010] [17:29:02] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 02 September 2010] [17:52:01] Quit gavinstark has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [18:03:02] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 02 September 2010] [18:31:17] <ModusPwnens> ok, so it still crashes
[Thursday 02 September 2010] [18:31:22] <ModusPwnens> i will paste the output in as ec
[Thursday 02 September 2010] [18:33:20] <ModusPwnens> I am using windows 7 on both computers
[Thursday 02 September 2010] [18:33:36] <ModusPwnens> The computer that generated the error is 64-bit as well
[Thursday 02 September 2010] [18:34:29] <ModusPwnens> the computer running local is only 32-bit
[Thursday 02 September 2010] [19:43:18] Quit poswald has left this server (Quit: poswald).
[Thursday 02 September 2010] [19:47:56] Quit poswald has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [20:07:47] <bgranger> sustrik: Just played with the EINTR logic.
[Thursday 02 September 2010] [20:31:13] Quit ModusPwnens has left this server.
[Thursday 02 September 2010] [20:35:38] Quit jonrafkind has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [21:25:39] Quit xrfang has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [21:46:33] <halfie> hi, does zeromq supports building of read-only queues with one writer and multiple readers?
[Thursday 02 September 2010] [21:47:38] Quit rossij has left this server (Ping timeout: 245 seconds).
[Thursday 02 September 2010] [22:10:13] Quit gebi has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [22:14:20] <halfie> ls
[Thursday 02 September 2010] [22:26:17] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [22:37:15] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:08:08] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:17:00] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:26:55] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:29:33] Quit rbraley has left this server (
*.net
*.split).
[Thursday 02 September 2010] [23:29:33] Quit GeekGod has left this server (
*.net
*.split).
[Thursday 02 September 2010] [23:38:49] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:48:43] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:56:19] Quit xrfang has left this server (Remote host closed the connection).
[Friday 03 September 2010] [00:31:22] Quit jsimmons has left this server (Quit: Leaving).
[Friday 03 September 2010] [00:39:25] Quit grahamalot has left this server (Quit: grahamalot).
[Friday 03 September 2010] [00:40:21] Join mikejs has joined this channel (
~me@mikej.st).
[Friday 03 September 2010] [01:51:52] Quit bgranger has left this server (Quit: bgranger).
[Friday 03 September 2010] [01:54:31] <guido_g> g’mornin’ (or whatever :)
[Friday 03 September 2010] [02:20:51] <sustrik> guido_g: thanks, i’ll have a look
[Friday 03 September 2010] [02:20:54] <sustrik> morning
[Friday 03 September 2010] [02:21:01] <sjampoo> I am kinda sad to see that with each update I find it more difficult to navigate around zeromq.??? :(
[Friday 03 September 2010] [02:22:05] <sustrik> sjampoo: i thought pieter reverted it to the original state
[Friday 03 September 2010] [02:22:05] <sustrik> let me see
[Friday 03 September 2010] [02:22:15] <sustrik> you mean the news should be in the right pane?
[Friday 03 September 2010] [02:22:28] <sjampoo> yes, at the least
[Friday 03 September 2010] [02:22:54] <sustrik> if you have any idea what to do further, just shoot
[Friday 03 September 2010] [02:24:21] <sjampoo> first thing would be not to change it every month ;) and I send an email about it months ago to the list. I also do not really like the .com / .org split
[Friday 03 September 2010] [02:24:49] <sjampoo> If the .org site is difficult to understand for new users, just make it more easy to understand
[Friday 03 September 2010] [02:25:04] <sustrik> where would you put the community stuff?
[Friday 03 September 2010] [02:25:20] <sjampoo> I would hide that on a separate but consistent page
[Friday 03 September 2010] [02:25:33] <sustrik> www.zeromq.org/dev say?
[Friday 03 September 2010] [02:25:42] <sjampoo> Yes, something like that.
[Friday 03 September 2010] [02:26:00] <sjampoo> I mean only 0.005% is interested what the actual last edited page is.
[Friday 03 September 2010] [02:26:22] <sjampoo> Most people interested in the community, will most likely just want to see GITHUB or the IssueList
[Friday 03 September 2010] [02:26:35] <sustrik> no mailing list?
[Friday 03 September 2010] [02:26:56] <sjampoo> and the mailinglist
[Friday 03 September 2010] [02:27:06] <sjampoo> But I don’t think that should be on the frontpage
[Friday 03 September 2010] [02:27:40] <sustrik> wouldn’t you mind clicking through a simple page each time to the dev page?
[Friday 03 September 2010] [02:28:20] <sjampoo> No, not if it is at a consistent location
[Friday 03 September 2010] [02:28:25] <sjampoo> I could bookmark it
[Friday 03 September 2010] [02:28:29] <sustrik> right
[Friday 03 September 2010] [02:28:42] <sustrik> in summary:
[Friday 03 September 2010] [02:28:50] <sustrik> 1. move zeromq.com to zeromq.org
[Friday 03 September 2010] [02:28:59] <sustrik> 2. move zeromq.org to zeromq.org/community
[Friday 03 September 2010] [02:29:10] <sustrik> 3. move the news to the right pane
[Friday 03 September 2010] [02:29:29] <sustrik> 4. get rid of recent site changes
[Friday 03 September 2010] [02:29:37] <sustrik> that’s it?
[Friday 03 September 2010] [02:30:15] <sjampoo> I am not completely sure what you mean with moving .com to .org.
[Friday 03 September 2010] [02:30:20] <sjampoo> I am not a big fan of the .com design
[Friday 03 September 2010] [02:30:36] <sustrik> that’s because you are part of community
[Friday 03 September 2010] [02:30:44] <sjampoo> No, no
[Friday 03 September 2010] [02:31:06] <sjampoo> I just don’t think that people landing on that site will get a good idea about what 0mq is
[Friday 03 September 2010] [02:31:20] <sustrik> ok, here’s the use case:
[Friday 03 September 2010] [02:31:21] <sjampoo> It needs a little more meat
[Friday 03 September 2010] [02:31:46] <sustrik> you have a problem and someone advises you 0mq may help to solve it
[Friday 03 September 2010] [02:31:55] <sustrik> you type zeromq into goodle
[Friday 03 September 2010] [02:31:58] <sustrik> google
[Friday 03 September 2010] [02:32:15] <sustrik> which, ideally, would bring you to what’s now zeromq.com
[Friday 03 September 2010] [02:32:27] <sustrik> you are under time stress
[Friday 03 September 2010] [02:32:36] <sustrik> so you cannot browse around much
[Friday 03 September 2010] [02:32:41] <sustrik> what you need to know:
[Friday 03 September 2010] [02:32:56] <sustrik> 1. what is it about (very roughly)
[Friday 03 September 2010] [02:33:00] <sustrik> 2. does it run on my OS
[Friday 03 September 2010] [02:33:09] <sustrik> 3. does it work with my language
[Friday 03 September 2010] [02:33:37] <sustrik> 4. i want to see a simple code example just to make sure it’s not a utter shit
[Friday 03 September 2010] [02:33:49] <sustrik> after that i want a one click download
[Friday 03 September 2010] [02:34:09] <sjampoo> All that is true, but you are missing the biggest yet.
[Friday 03 September 2010] [02:34:15] <sustrik> yes?
[Friday 03 September 2010] [02:34:16] <sjampoo> “Why should I care”
[Friday 03 September 2010] [02:34:27] <sustrik> because i have a problem to solve
[Friday 03 September 2010] [02:34:37] <sustrik> i have to deliver tomorrow
[Friday 03 September 2010] [02:34:52] <sustrik> network part is still missing :(
[Friday 03 September 2010] [02:34:53] <sustrik> etc.
[Friday 03 September 2010] [02:35:11] <sjampoo> Yes, but that isn’t explained currently on zmq.com
[Friday 03 September 2010] [02:35:27] <sustrik> it can be improved obviously
[Friday 03 September 2010] [02:35:39] <sustrik> but the main point is that most people are not browsing for fun
[Friday 03 September 2010] [02:35:44] <sustrik> they are looking for solution
[Friday 03 September 2010] [02:36:16] <sustrik> so what they ask is “would this solve my problem” rather then “what it is good for”
[Friday 03 September 2010] [02:36:24] <sjampoo> Absolutetely. Ok, I would really add a link sidebar on the left side of zeromq.com
[Friday 03 September 2010] [02:36:58] <sustrik> what links?
[Friday 03 September 2010] [02:38:16] <sjampoo> all the ‘content’ links on zeromq.org, links to Github, Blog and Mailinglist
[Friday 03 September 2010] [02:38:40] <sustrik> too confusing imo
[Friday 03 September 2010] [02:39:09] <sustrik> that’s good once you are using the thing
[Friday 03 September 2010] [02:39:34] <sustrik> the zeromq.com is there just to streamline the process up to the dowload phase
[Friday 03 September 2010] [02:40:35] <sustrik> once you’re developing with 0mq you would naturally switch to the community site
[Friday 03 September 2010] [02:41:34] * lestrrat didn’t even know there was a zeromq.com
[Friday 03 September 2010] [02:41:57] <sustrik> it’s new
[Friday 03 September 2010] [02:42:06] <sustrik> created day before yesterdat
[Friday 03 September 2010] [02:43:23] <sustrik> sjampoo: i think you are right about .org, it’s less readable now
[Friday 03 September 2010] [02:43:36] <sustrik> but .com (or however it is called) has it meaning
[Friday 03 September 2010] [02:43:57] <sustrik> althought the user it is aimed at are invisible to you – visible through google analytics though
[Friday 03 September 2010] [02:44:37] <sjampoo> sustrik: I can see what it tries to do, but I am afraid that zeromq.com in its current state will not leave the user with a ‘WOW’ feeling.
[Friday 03 September 2010] [02:44:56] <sustrik> that’s not it goal i think
[Friday 03 September 2010] [02:44:57] <sjampoo> Atleast get some sort of ‘Who is already using zeromq’
[Friday 03 September 2010] [02:45:08] <sustrik> hm
[Friday 03 September 2010] [02:45:23] <sustrik> do you look at that kind of think when inspecting unknown projects?
[Friday 03 September 2010] [02:45:35] <sjampoo> sustrik: absolutely
[Friday 03 September 2010] [02:45:42] <sjampoo> Is it worth my time to investigate
[Friday 03 September 2010] [02:45:48] <sjampoo> will it gain me anything?
[Friday 03 September 2010] [02:45:54] <sjampoo> Or is it something flunky and experimental
[Friday 03 September 2010] [02:46:06] <sustrik> it may be part of “making sure it’s not a shit” step
[Friday 03 September 2010] [02:46:17] <sjampoo> exactly
[Friday 03 September 2010] [02:47:18] <sustrik> what about the “introductions”?
[Friday 03 September 2010] [02:47:23] <sustrik> not good enough?
[Friday 03 September 2010] [02:47:55] <sjampoo> well, I was more thinking about some company profiles that are currently using zeromq
[Friday 03 September 2010] [02:48:52] <sjampoo> And it really should be put on the front page, especially if you take the (no doubt high) bounce rate into account.
[Friday 03 September 2010] [02:49:06] <sustrik> does reading a list of company names you never heard about before make you more happy then quotes?
[Friday 03 September 2010] [02:49:16] <sjampoo> hehe
[Friday 03 September 2010] [02:49:16] <sustrik> (i’m not trolling, i’m really interested)
[Friday 03 September 2010] [02:49:26] <sjampoo> sustrik: interesting question
[Friday 03 September 2010] [02:49:33] <sjampoo> no, I think both are well
[Friday 03 September 2010] [02:49:38] <sjampoo> uh i mean will do
[Friday 03 September 2010] [02:49:48] <sjampoo> And I am pretty sure that it doesn’t really matter who says what
[Friday 03 September 2010] [02:50:07] <sjampoo> that’s why astroturfing is so succesful.
[Friday 03 September 2010] [02:50:18] <sjampoo> We people are so sensitive to what others think
[Friday 03 September 2010] [02:50:46] <sustrik> yeah, but its unlikely to get madonna say anything about 0mq
[Friday 03 September 2010] [02:50:57] <sustrik> so you are still stuck with quotes from people you don’t know
[Friday 03 September 2010] [02:51:19] <sjampoo> exactly, and that doesn’t matter
[Friday 03 September 2010] [02:51:28] <sustrik> yes, quote just sounds well
[Friday 03 September 2010] [02:51:40] <sustrik> because someone invested 10 seconds to write it
[Friday 03 September 2010] [02:52:12] <sustrik> company name is less personal
[Friday 03 September 2010] [02:53:13] <sustrik> now, as for moving it to front page
[Friday 03 September 2010] [02:53:27] <sustrik> do you read endorsements first?
[Friday 03 September 2010] [02:53:47] <sustrik> even before what the project is and whether it will run with your OS/language
[Friday 03 September 2010] [02:54:07] * sustrik is doing it other way round
[Friday 03 September 2010] [02:56:12] <sjampoo> I usually read from top top bottom, but sometime I get distracted by bigger and bolder fonts.
[Friday 03 September 2010] [02:56:26] <sjampoo> and when your lucky I actually click on a link instead of back
[Friday 03 September 2010] [02:56:35] <sustrik> hm
[Friday 03 September 2010] [02:56:53] <sustrik> what i do is a type keywords into google bar
[Friday 03 September 2010] [02:57:03] <sustrik> i get a list of more or less matching projects
[Friday 03 September 2010] [02:57:13] <guido_g> re
[Friday 03 September 2010] [02:57:21] <sustrik> then i look at each one discarding those that won’t fit
[Friday 03 September 2010] [02:57:27] <sjampoo> sustrik: yes, exactly
[Friday 03 September 2010] [02:57:28] <guido_g> ahhh web site issues :)
[Friday 03 September 2010] [02:57:32] <sustrik> i spend say 30 seconds on each page
[Friday 03 September 2010] [02:57:43] <guido_g> could one please reconsider the size of the logo?
[Friday 03 September 2010] [02:57:53] <sustrik> do you want it bigger?
[Friday 03 September 2010] [02:57:55] <sustrik> :)
[Friday 03 September 2010] [02:57:59] <guido_g> when viewed on small screen its taking too much space
[Friday 03 September 2010] [02:58:09] * sustrik makes a note
[Friday 03 September 2010] [02:58:11] <guido_g> sustrik: bad boy! :)
[Friday 03 September 2010] [02:58:28] <guido_g> it’s more than 1/3 of vertical screen space
[Friday 03 September 2010] [02:58:36] <sjampoo> For a page to appear in Google it needs to be indexed with the right terms.
[Friday 03 September 2010] [02:58:38] <sustrik> what’s you screen size?
[Friday 03 September 2010] [02:58:42] <sustrik> just for reference
[Friday 03 September 2010] [02:58:57] <guido_g> and this color makes one aggressive...
[Friday 03 September 2010] [02:59:05] <guido_g> 13.6’‘
[Friday 03 September 2010] [02:59:14] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [02:59:14] <sustrik> i meant in pixels
[Friday 03 September 2010] [02:59:15] <sjampoo> guido_g: ah yes colors
[Friday 03 September 2010] [02:59:19] <sjampoo> lets discuss that next
[Friday 03 September 2010] [02:59:30] * sjampoo grins
[Friday 03 September 2010] [02:59:42] <sjampoo> I actually like the red big logo
[Friday 03 September 2010] [02:59:57] <sustrik> yes, i thought of red logo + shades of grey underneath
[Friday 03 September 2010] [03:00:09] <guido_g> 1280x800
[Friday 03 September 2010] [03:00:15] <sjampoo> it would make a nice tattoo
[Friday 03 September 2010] [03:00:15] <sustrik> it’s kind of dull, but maybe hurts eyes less
[Friday 03 September 2010] [03:00:30] <sustrik> guido_g: thanks
[Friday 03 September 2010] [03:01:06] <sustrik> let’s finish the content thing:
[Friday 03 September 2010] [03:01:08] <guido_g> it’s not that bad when using 1920x1200 on a 27’’ display
[Friday 03 September 2010] [03:01:26] <sustrik> so the front page at zeromq.com is meant to convey, in 30 secs
[Friday 03 September 2010] [03:01:34] <guido_g> but still, takes to much space that could’ve been used for better things
[Friday 03 September 2010] [03:01:39] <sustrik> whether the project is usable in your particular case or not
[Friday 03 September 2010] [03:01:54] <sustrik> guido_g: yes, we can do some experimenting with it
[Friday 03 September 2010] [03:02:37] <guido_g> if it should be for new users, why must i have cookies allowed then?
[Friday 03 September 2010] [03:02:57] <sustrik> what does it do if you don’t?
[Friday 03 September 2010] [03:03:13] <guido_g> it shows a message to activate cookies
[Friday 03 September 2010] [03:03:21] <sustrik> yuck
[Friday 03 September 2010] [03:03:33] * sustrik is not a web person, let me ask pieter
[Friday 03 September 2010] [03:03:33] <guido_g> Can’t proceed, you should accept cookies for this domain.
[Friday 03 September 2010] [03:04:23] <guido_g> i mean, i know mq, so i’ll activate cookies
[Friday 03 September 2010] [03:04:45] <guido_g> but if this would have been the first impression, i’d closed the tab
[Friday 03 September 2010] [03:05:07] <sustrik> right, i am not sure what may be the case. if it’s wikidot requiring the cookies we can move the page elsewhere
[Friday 03 September 2010] [03:05:20] <guido_g> i think it’s wikidot
[Friday 03 September 2010] [03:05:23] <sustrik> it doesn’t have to be a wiki after all
[Friday 03 September 2010] [03:05:42] <guido_g> because you need to allow cross-domain cookies (or how they’re called)
[Friday 03 September 2010] [03:06:01] <sustrik> does that happen with .org as well?
[Friday 03 September 2010] [03:06:16] <guido_g> a community site should be accessible even for users taking their privacy serious
[Friday 03 September 2010] [03:06:25] <guido_g> i’ll take a look
[Friday 03 September 2010] [03:06:46] <guido_g> yes
[Friday 03 September 2010] [03:08:10] <guido_g> btw, wikidot cookies
[Friday 03 September 2010] [03:09:39] <sustrik> hm, i switched off cookies and it still works for me
[Friday 03 September 2010] [03:09:50] <sustrik> what browser are you using?
[Friday 03 September 2010] [03:10:00] <guido_g> btw, i think for the community site we don’t need a large funny picture at the top of the entry page
[Friday 03 September 2010] [03:10:09] <guido_g> opera 10.60
[Friday 03 September 2010] [03:11:58] <sustrik> hm
[Friday 03 September 2010] [03:12:14] <guido_g> i don’t like the sound of “hm” :)
[Friday 03 September 2010] [03:12:52] <sustrik> i was just wondering if i should install opera :)
[Friday 03 September 2010] [03:13:05] <guido_g> hehe
[Friday 03 September 2010] [03:13:09] <sustrik> but let’s rather ask at wikidot
[Friday 03 September 2010] [03:13:14] * sustrik makes another note
[Friday 03 September 2010] [03:13:24] <guido_g> ok, thx for listening
[Friday 03 September 2010] [03:13:47] * guido_g is off to $day-job
[Friday 03 September 2010] [03:13:52] <sustrik> thanks for the feedback
[Friday 03 September 2010] [03:14:01] <guido_g> no problem
[Friday 03 September 2010] [03:14:13] Quit guido_g has left this server (Quit: Boom...).
[Friday 03 September 2010] [03:23:45] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 03 September 2010] [03:36:41] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 03 September 2010] [03:39:08] Quit xrfang has left this server (Remote host closed the connection).
[Friday 03 September 2010] [03:47:33] <guido_g> re
[Friday 03 September 2010] [03:52:29] Nick vic is now known as Guest26371.
[Friday 03 September 2010] [04:13:34] <sustrik> guido_g: the article at acm queue is good
[Friday 03 September 2010] [04:13:45] <sustrik> i wanted to write something like this for a long time
[Friday 03 September 2010] [04:13:53] <sustrik> but never had enough time to actually do it
[Friday 03 September 2010] [04:14:25] <guido_g> oh, so another dream died :)
[Friday 03 September 2010] [04:14:30] <
halfie_> hi, can I modify the publish-subscribe model so that the “queue” is read-only?
[Friday 03 September 2010] [04:14:32] <
halfie_> I don
[Friday 03 September 2010] [04:14:53] <guido_g> huh?
[Friday 03 September 2010] [04:14:57] <guido_g> what does that mean?
[Friday 03 September 2010] [04:15:04] <guido_g> SUB sockets are read only...
[Friday 03 September 2010] [04:15:41] <
halfie_> guido_g, I mean that the subscriber shouldn’t remove the message
[Friday 03 September 2010] [04:15:57] <sustrik> you need a database, not a messaging system
[Friday 03 September 2010] [04:16:18] <sustrik> publisher -> INSERT
[Friday 03 September 2010] [04:16:19] <guido_g> ack
[Friday 03 September 2010] [04:16:23] <sustrik> subscriber -> SELECT
[Friday 03 September 2010] [04:17:01] <
halfie_> I see, makes sense.
[Friday 03 September 2010] [04:17:31] <
halfie_> some kind of in-memory database with “queue” semantics would solve the problem
[Friday 03 September 2010] [04:18:11] <guido_g> berkeley db
[Friday 03 September 2010] [04:19:12] <mato> ‘morning
[Friday 03 September 2010] [04:19:50] <
halfie_> guido_g, thanks, I will give it a shot. sustrik: thanks for clearing the situation :)
[Friday 03 September 2010] [04:20:20] <sustrik> :)
[Friday 03 September 2010] [04:20:27] <sustrik> mato: morning
[Friday 03 September 2010] [04:20:34] <sustrik> a question
[Friday 03 September 2010] [04:20:38] <mato> yup?
[Friday 03 September 2010] [04:20:39] <sustrik> a signal question!
[Friday 03 September 2010] [04:20:40] <guido_g> hi mato
[Friday 03 September 2010] [04:20:45] <mato> hi guido
[Friday 03 September 2010] [04:20:51] <sustrik> imagine this:
[Friday 03 September 2010] [04:21:03] <sustrik> binding calls 0mq’s blocking function
[Friday 03 September 2010] [04:21:20] <sustrik> 99% of the time is spent in OS’s blocking call
[Friday 03 September 2010] [04:21:32] <sustrik> 1% of time some 0mq processing is being done
[Friday 03 September 2010] [04:21:46] <sustrik> what if user Ctrl+C’s during the 1%?
[Friday 03 September 2010] [04:21:55] <mato> too bad
[Friday 03 September 2010] [04:21:58] <sustrik> :)
[Friday 03 September 2010] [04:22:02] <mato> it’ll get delivered late
[Friday 03 September 2010] [04:22:06] <mato> nothing you can do about that
[Friday 03 September 2010] [04:22:23] <mato> well, there are various weird hacks you can do about that, but believe me, you don’t want to go there
[Friday 03 September 2010] [04:22:29] <sustrik> it won’t be delivered at all no?
[Friday 03 September 2010] [04:22:50] <sustrik> the handler runs, sets a bit
[Friday 03 September 2010] [04:23:02] <sustrik> there’s no blocking call for OS to interrupt
[Friday 03 September 2010] [04:23:06] <sustrik> so nothing happens
[Friday 03 September 2010] [04:23:11] <mato> hmm
[Friday 03 September 2010] [04:23:16] <sustrik> 0mq then calls a blocking call
[Friday 03 September 2010] [04:23:20] <sustrik> and freezes
[Friday 03 September 2010] [04:23:47] <mato> oh dear
[Friday 03 September 2010] [04:23:50] <mato> yes, you may be right
[Friday 03 September 2010] [04:24:02] <sustrik> the whole EINTR thing seems broken
[Friday 03 September 2010] [04:24:06] <mato> i have always said signal handling is broken
[Friday 03 September 2010] [04:24:10] <mato> to put it very politely
[Friday 03 September 2010] [04:24:16] <sustrik> is there any way to make it work?
[Friday 03 September 2010] [04:24:31] <mato> hmm, well, maybe
[Friday 03 September 2010] [04:24:42] <sustrik> imagine you are omnipotent
[Friday 03 September 2010] [04:24:44] <mato> but it’s so horribly racy i wouldn’t want to go there
[Friday 03 September 2010] [04:24:56] <mato> oh, if i’m omnipotent then 0mq is in kernel space, problem goes away
[Friday 03 September 2010] [04:24:57] <sustrik> you can change the language runtimes etc.
[Friday 03 September 2010] [04:25:08] <mato> ah, less than god (linus), ok...
[Friday 03 September 2010] [04:25:11] <mato> just a sub-god :)
[Friday 03 September 2010] [04:25:22] <sustrik> what i’m asking: is there a canonical way to use EINTR
[Friday 03 September 2010] [04:25:28] <sustrik> something that works?
[Friday 03 September 2010] [04:25:31] <mato> well, there is a canonical way i would do it
[Friday 03 September 2010] [04:25:40] <mato> this is one of the ideas i had yesterday
[Friday 03 September 2010] [04:25:44] <mato> it might not be right
[Friday 03 September 2010] [04:25:48] * sustrik is listening
[Friday 03 September 2010] [04:25:50] <mato> but assuming you can hack the language runtime
[Friday 03 September 2010] [04:26:36] <mato> then what you need to do is chuck a call to a thread-safe “zmq_interrupt()” into the low-level signal handler in the runtime
[Friday 03 September 2010] [04:26:41] <mato> i.e. the bit that “sets a flag”
[Friday 03 September 2010] [04:26:58] <mato> this zmq_interrupt() would cause all application API calls to immediately return with EINTR
[Friday 03 September 2010] [04:27:13] <mato> it’s ugly, but it should work
[Friday 03 September 2010] [04:27:37] <sustrik> tweaking it a bit you end up with chained signal handlers
[Friday 03 September 2010] [04:27:49] <mato> yes, except there’s no API for that
[Friday 03 September 2010] [04:27:53] <sustrik> exactly
[Friday 03 September 2010] [04:27:59] <mato> so you don’t want to go there :)
[Friday 03 September 2010] [04:28:08] <mato> trying to emulate chained signal handlers is madness
[Friday 03 September 2010] [04:28:16] <sustrik> so the whole system is really broken
[Friday 03 September 2010] [04:28:20] <mato> yes
[Friday 03 September 2010] [04:28:46] <sustrik> ok, what i do is froward the EINTR to the binding
[Friday 03 September 2010] [04:28:49] <mato> it doesn’t account for the fact that more than one “entity” inside the application might want to know if a signal arrived
[Friday 03 September 2010] [04:28:54] <sustrik> that would work in99% of cases
[Friday 03 September 2010] [04:29:10] <sustrik> the rest...
[Friday 03 September 2010] [04:29:11] <mato> “forward the EINTR to the binding”? how?
[Friday 03 September 2010] [04:29:11] <sustrik> shrug
[Friday 03 September 2010] [04:29:25] <sustrik> if (errno == EINTR) return -1;
[Friday 03 September 2010] [04:29:30] <mato> oh, that, yes...
[Friday 03 September 2010] [04:29:34] <mato> hmm
[Friday 03 September 2010] [04:29:39] <mato> let me check something
[Friday 03 September 2010] [04:31:34] Join plq has joined this channel (
~plq@78.189.30.218).
[Friday 03 September 2010] [04:34:32] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 03 September 2010] [04:44:26] <mato> sustrik: ok, so, yes, forwarding EINTR is the only way at the moment
[Friday 03 September 2010] [04:44:33] <sustrik> ack
[Friday 03 September 2010] [04:44:41] <sustrik> well, there’s another wat
[Friday 03 September 2010] [04:44:43] <sustrik> way
[Friday 03 September 2010] [04:44:45] <mato> yes?
[Friday 03 September 2010] [04:45:03] <sustrik> on each entry to potentially blocking 0mq function
[Friday 03 September 2010] [04:45:11] <sustrik> store the old signal handler
[Friday 03 September 2010] [04:45:14] <sustrik> install your own
[Friday 03 September 2010] [04:45:15] <sustrik> etc.
[Friday 03 September 2010] [04:45:25] <sustrik> then on exit return it to the original state
[Friday 03 September 2010] [04:45:27] <mato> that’s the naive way, yes
[Friday 03 September 2010] [04:45:29] <mato> except
[Friday 03 September 2010] [04:45:35] <mato> IT WILL NEVER WORK :)
[Friday 03 September 2010] [04:45:41] <sustrik> why so?
[Friday 03 September 2010] [04:46:09] <mato> because, and this is what i just double-checked, all the POSIX docs explicitly say “never, ever, ever, try to mix different signal handling mechanisms and threads”
[Friday 03 September 2010] [04:46:25] <mato> phone, hang on
[Friday 03 September 2010] [04:46:56] <sustrik> not sure what that means, but ok, i don’t want to implement that kind of hack anyway
[Friday 03 September 2010] [04:48:28] <mato> let me explain
[Friday 03 September 2010] [04:48:40] <mato> the problem is it only works if you have 1 thread
[Friday 03 September 2010] [04:48:41] <mato> 1 OS thread
[Friday 03 September 2010] [04:48:45] <mato> and no more
[Friday 03 September 2010] [04:49:18] <mato> if you start doing that kind of stuff it will break any application that might actually trying to be do things the right way in a MT envinronment, which is using a sigwait() thread
[Friday 03 September 2010] [04:49:29] <mato> you have no way of knowing what the application is doing...
[Friday 03 September 2010] [04:49:34] <mato> does that make sense?
[Friday 03 September 2010] [04:50:00] <sustrik> i have no idea how sigwait style interacts with the classic style
[Friday 03 September 2010] [04:50:13] <mato> badly
[Friday 03 September 2010] [04:50:16] <mato> i.e. never do that
[Friday 03 September 2010] [04:50:16] <sustrik> but i don’t really care
[Friday 03 September 2010] [04:50:30] <sustrik> so for now i’ll return EINTR
[Friday 03 September 2010] [04:50:31] <mato> yeah, but if you start mucking with signal handlers then you start caring :)
[Friday 03 September 2010] [04:50:33] <sustrik> when available
[Friday 03 September 2010] [04:50:37] <mato> yes
[Friday 03 September 2010] [04:50:49] <mato> returning EINTR will solve the problem “most of the time”
[Friday 03 September 2010] [04:50:50] <sustrik> so sometimes Ctrl+C won’t work
[Friday 03 September 2010] [04:50:53] Quit jsimmons has left this server (Quit: Leaving).
[Friday 03 September 2010] [04:50:58] <mato> but you can still get signal loss someimes
[Friday 03 September 2010] [04:51:01] <sustrik> however, user can press Ctrl+C again
[Friday 03 September 2010] [04:51:11] <mato> this just needs to be documented, that’s all
[Friday 03 September 2010] [04:51:35] * mato sees the 0MQ FAQ on signal handling becoming into a treatise on the brokenness of signals :)
[Friday 03 September 2010] [04:52:03] <sustrik> yeah
[Friday 03 September 2010] [04:52:11] <mato> sustrik: anyway, can i suggest the following process for this:
[Friday 03 September 2010] [04:52:28] <mato> sustrik: 1. Test the solution, as discussed yesterday.
[Friday 03 September 2010] [04:52:43] <mato> sustrik: 2. Post a patch for review, along with your “I’m (almost) an idiot, sorry”
[Friday 03 September 2010] [04:53:19] <mato> sustrik: 3. I will reply to that describing all I know about the problem so that the binding people can understand what’s going on (since they’re the most affected)
[Friday 03 September 2010] [04:53:56] <mato> sustrik: 4. Some consensus will be reached on whether or not this goes into 2.0.x (I’m not opposed either way, it’s a fix for breakage on our part)
[Friday 03 September 2010] [04:54:30] <mato> sustrik: 5. We may learn something useful from the discussion, and I can also bring up the zmq_interrupt() idea which might give a solution for those language runtimes that want to solve this 100%...
[Friday 03 September 2010] [04:54:41] <mato> sustrik: what do you think?
[Friday 03 September 2010] [04:56:31] <mato> phone call, back in 20mins
[Friday 03 September 2010] [04:56:33] <pieterh> re
[Friday 03 September 2010] [04:56:40] <pieterh> i lost my phone and have been offline
[Friday 03 September 2010] [04:56:42] <pieterh> :-(
[Friday 03 September 2010] [04:58:02] <pieterh> sjampoo: i read your remarks about the website
[Friday 03 September 2010] [04:58:11] <pieterh> they make sense and we’ll evolve this over time
[Friday 03 September 2010] [04:59:50] Quit
gebi_ has left this server (Ping timeout: 272 seconds).
[Friday 03 September 2010] [05:20:39] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 03 September 2010] [05:23:29] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [05:30:36] Quit poswald has left this server (Quit: poswald).
[Friday 03 September 2010] [05:35:23] Quit poswald has left this server (Ping timeout: 260 seconds).
[Friday 03 September 2010] [05:46:21] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Friday 03 September 2010] [05:54:53] <mato> sustrik: re
[Friday 03 September 2010] [05:54:59] <mato> sustrik: are you happy with what i propose?
[Friday 03 September 2010] [05:55:15] <mato> sustrik: it seems like the best way to move forward on the signal handling issues
[Friday 03 September 2010] [07:00:57] <sustrik> mato: yes
[Friday 03 September 2010] [07:01:00] <sustrik> re
[Friday 03 September 2010] [07:01:04] <mato> sustrik: see my long email
[Friday 03 September 2010] [07:01:13] <mato> sustrik: i’ve tried to explain the issue, complex as it is
[Friday 03 September 2010] [07:01:27] <sustrik> ok, i’m on it
[Friday 03 September 2010] [07:08:05] Quit guido_g has left this server (Quit: Boom...).
[Friday 03 September 2010] [07:16:43] Join pieterh has joined this channel (
~ph@217.75.64.166).
[Friday 03 September 2010] [07:30:29] <mato> ok, another big long email sent re the Python GIL stuff
[Friday 03 September 2010] [07:30:44] <mato> somewhat braindump format, hopefully it’s clear enough
[Friday 03 September 2010] [08:12:23] <arnorhs> yo
[Friday 03 September 2010] [08:12:42] <arnorhs> Which programming language has the best zero mq bindings?
[Friday 03 September 2010] [08:12:54] <arnorhs> Which one is the most used / most stable?
[Friday 03 September 2010] [08:15:53] <guido_g> mq itself is programmin in c++, so this should work
[Friday 03 September 2010] [08:16:08] <guido_g> c is delivered with the core, so does work too
[Friday 03 September 2010] [08:16:38] <sustrik> yes, all the bindings use C API underneath
[Friday 03 September 2010] [08:16:42] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [08:17:04] <guido_g> the other bindings are quite stable, if you don’t insinst on using bleeding edge versions
[Friday 03 September 2010] [08:26:55] <arnorhs> ok, thanks
[Friday 03 September 2010] [08:27:03] <arnorhs> I will go for fortran then
[Friday 03 September 2010] [08:27:11] <arnorhs> :)
[Friday 03 September 2010] [08:27:24] <arnorhs> I’m guessing the python bindings are fine?
[Friday 03 September 2010] [08:30:17] <sustrik> yes, i think they are most stable among the whole bunch
[Friday 03 September 2010] [08:51:24] <arnorhs> ok, thanks
[Friday 03 September 2010] [08:53:11] <gebi> are there any plans to add {active, once} mode to the erlang bindings?
[Friday 03 September 2010] [08:53:35] Quit arnorhs has left this server (Remote host closed the connection).
[Friday 03 September 2010] [08:58:54] <sustrik> gebi: what’s once?
[Friday 03 September 2010] [09:23:46] Quit plq has left this server (Remote host closed the connection).
[Friday 03 September 2010] [09:30:37] <gebi> sustrik: only deliver ONE message
[Friday 03 September 2010] [09:31:07] <gebi> with active,true is unuseable because it will fill up your process message queue and make it unusable
[Friday 03 September 2010] [09:31:45] <gebi> active,false is also not ideal because processes block and do not process other messages
[Friday 03 September 2010] [10:31:44] <sustrik> gebi: how does that work
[Friday 03 September 2010] [10:31:50] <sustrik> ok, it delivers one message
[Friday 03 September 2010] [10:31:55] <sustrik> when then?
[Friday 03 September 2010] [10:32:10] <sustrik> you still have to have a way to get next message
[Friday 03 September 2010] [10:32:20] <sustrik> mato: are you here?
[Friday 03 September 2010] [10:35:26] <sustrik> gebi: does it mean there’s at most one message in erlang queue?
[Friday 03 September 2010] [10:35:42] <sustrik> if so, how do you tell erlang to behave that way?
[Friday 03 September 2010] [10:59:09] <gebi> sustrik: it should be a property of the erlang 0mq binding
[Friday 03 September 2010] [11:00:20] <sustrik> :)
[Friday 03 September 2010] [11:00:37] <sustrik> i know, i was just interested whether erlnag as such has such feature
[Friday 03 September 2010] [11:00:41] <gebi> with active, once it just sends delivers a single received datagram into the message queue and stops then
[Friday 03 September 2010] [11:00:42] <sustrik> erlang*
[Friday 03 September 2010] [11:01:03] <sustrik> that’s erlang’s native networking API?
[Friday 03 September 2010] [11:01:49] <gebi> yea it’s a property of gen_tcp
[Friday 03 September 2010] [11:02:16] * sustrik is checking it
[Friday 03 September 2010] [11:03:13] <gebi> you just call inet:setopts(S,[{active,once}]) when you want to get another datagram delivered into your messagequeue
[Friday 03 September 2010] [11:04:25] <sustrik> aha
[Friday 03 September 2010] [11:04:33] <sustrik> how is that different from passive?
[Friday 03 September 2010] [11:04:55] <gebi> what do you mean with passive?
[Friday 03 September 2010] [11:05:11] <sustrik> active=false
[Friday 03 September 2010] [11:05:23] Quit dos000 has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [11:05:23] Quit sachinr has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [11:06:34] <gebi> with active false you have an effectively dead process (all your receiving processes hang on erlzmq:receive)
[Friday 03 September 2010] [11:07:36] <gebi> with all the bad sideeffects
[Friday 03 September 2010] [11:08:20] <sustrik> ah, it blocks threads in erlnag thread pool?
[Friday 03 September 2010] [11:08:25] <sustrik> erlnag*
[Friday 03 September 2010] [11:08:28] <sustrik> erlang*
[Friday 03 September 2010] [11:08:34] <gebi> no erlang processes
[Friday 03 September 2010] [11:09:15] <sustrik> does it matter? if process is up to receiving a message am there’s no message available it hangs anyway
[Friday 03 September 2010] [11:09:55] <gebi> if it “hangs” on receiving a erlang message it is alive, if it hangs on receiving a message from the 0mq binding it is dead
[Friday 03 September 2010] [11:10:23] <sustrik> does “dead” mean that it is ubable to process erlang messages?
[Friday 03 September 2010] [11:10:25] <gebi> meaning it wont process erlang messages till at least _one_ 0mq message gets delivered (and erlzmq:recv returns)
[Friday 03 September 2010] [11:10:32] <sustrik> ok, got it
[Friday 03 September 2010] [11:10:48] <sustrik> makes sense
[Friday 03 September 2010] [11:10:51] <gebi> this _is_ the fastest mode, though most of the time not desireable
[Friday 03 September 2010] [11:11:21] <sustrik> good, i’ll probably have a look at it next week
[Friday 03 September 2010] [11:11:26] <gebi> same with active,true, processing the message-queue in erlang is NOT O(1)
[Friday 03 September 2010] [11:11:39] <sustrik> ack
[Friday 03 September 2010] [11:11:57] <gebi> so it’s kinda ugly to have a big messagequeue, thus active,true is not possibel without another protocol above for flowcontrole
[Friday 03 September 2010] [11:12:00] <gebi> thx :)
[Friday 03 September 2010] [11:12:37] <sustrik> i was actually wondering how the flow control thing can be handled in erlang
[Friday 03 September 2010] [11:12:48] <sustrik> you’ve gave me the answer for free :)
[Friday 03 September 2010] [11:12:53] <gebi> heh, np :)
[Friday 03 September 2010] [11:31:58] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r9e7a9d1 10/ src/zmq.cpp : zmq_poll returns EINTR if signal interrupts it -
http://bit.ly/cWOXej[Friday 03 September 2010] [11:31:59] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * rb059116 10/ (9 files): EINTR is propogated from signaler_t::recv all the way up to the 0MQ API -
http://bit.ly/bOz5Ui[Friday 03 September 2010] [11:32:01] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r63706a9 10/ (doc/zmq_poll.txt doc/zmq_recv.txt doc/zmq_send.txt): documentation modified to reflect the EINTR change -
http://bit.ly/92QIrS[Friday 03 September 2010] [11:51:44] <cremes> sustrik: i get an assertion failure when running the EINTR branch
[Friday 03 September 2010] [11:55:25] <sustrik> cremes: let me see
[Friday 03 September 2010] [12:01:47] Quit
feroz_ has left this server (Quit:
feroz_).
[Friday 03 September 2010] [12:07:10] <sustrik> cremes: try now
[Friday 03 September 2010] [12:08:03] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r1bc4e9e 10/ src/app_thread.cpp : one more EINTR fix -
http://bit.ly/ciucLd[Friday 03 September 2010] [12:10:07] <cremes> sustrik: works
[Friday 03 September 2010] [12:11:06] <sustrik> you mean Ctrl+C stops the app?
[Friday 03 September 2010] [12:11:12] <cremes> yessir
[Friday 03 September 2010] [12:11:15] <sustrik> woo-hoo!
[Friday 03 September 2010] [12:11:19] <cremes> indeed!
[Friday 03 September 2010] [12:11:30] <sustrik> that’s a bug ~2 years old
[Friday 03 September 2010] [12:11:43] <sustrik> maybe 2 1/2
[Friday 03 September 2010] [12:11:58] <sustrik> until now nobody had any clue how to solve ti
[Friday 03 September 2010] [12:12:01] <cremes> wow, good catch
[Friday 03 September 2010] [12:12:02] <sustrik> great!
[Friday 03 September 2010] [12:12:26] <cremes> is this going into 2.0.9 or 2.1?
[Friday 03 September 2010] [12:12:30] <sustrik> let’s wait for bgranger and lestrrat to confirm it’s working first
[Friday 03 September 2010] [12:12:38] <cremes> sure thing
[Friday 03 September 2010] [12:12:46] <sustrik> i am afraid of pushing it to a stable version
[Friday 03 September 2010] [12:12:51] <cremes> i’ll drop a note to the ML to make my report “official”
[Friday 03 September 2010] [12:13:07] <sustrik> yes, good idea
[Friday 03 September 2010] [12:13:22] <cremes> yeah, i wrote to the ML with my reasoning for this change to go into 2.1 but not 2.0.x
[Friday 03 September 2010] [12:13:30] <sustrik> +1
[Friday 03 September 2010] [12:13:51] <sustrik> at least it’s a motivation for migration to 2.1 :)
[Friday 03 September 2010] [12:16:17] <cremes> oh yeah, i’m looking forward to the thread migration changes too
[Friday 03 September 2010] [12:16:31] <cremes> and the new FD_EVENT stuff
[Friday 03 September 2010] [12:18:49] <sustrik> it’s there to test already
[Friday 03 September 2010] [12:19:00] <sustrik> it may be a bit buggy
[Friday 03 September 2010] [12:19:02] Quit
dos000_ has left this server (Quit: Leaving).
[Friday 03 September 2010] [12:19:03] <sustrik> but still...
[Friday 03 September 2010] [12:19:42] <cremes> sustrik: i’m pretty busy building out a production app using 0mq 2.0.x so i haven’t had any time to test 2.1.x
[Friday 03 September 2010] [12:20:00] <cremes> when you get closer to a release, you should ping the bindings authors and ask us to test
[Friday 03 September 2010] [12:20:06] <sustrik> sure, just updating you on the status
[Friday 03 September 2010] [12:20:12] <cremes> cool, i appreciate it
[Friday 03 September 2010] [12:20:14] <sustrik> we’ll do some testing ourselves
[Friday 03 September 2010] [12:20:19] <sustrik> in the next few weeks
[Friday 03 September 2010] [12:20:28] <sustrik> then it’s binding autohr’s turn
[Friday 03 September 2010] [12:20:45] <cremes> do you have a time frame for the 2.1 release? 2-4 weeks? 4-6 weeks?
[Friday 03 September 2010] [12:22:09] <sustrik> hm, it’s up to mato, he’s going to do the testing
[Friday 03 September 2010] [12:22:30] <sustrik> bet yeah, something between 2 ans 6 weeks i would say
[Friday 03 September 2010] [12:27:05] <cremes> ok
[Friday 03 September 2010] [12:28:10] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [13:14:25] Quit xla has left this server (Quit: leaving).
[Friday 03 September 2010] [13:31:21] <ModusPwnens> So, I updated to 2.0.8 and I still have the problem of the benchmarking programs spitting out errors if you enter in very large parameters
[Friday 03 September 2010] [13:31:23] Quit gebi has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [13:32:07] <ModusPwnens> i’m using windows 7 on both operating systems, but the system running the remote end is 64-bit
[Friday 03 September 2010] [13:32:37] <ModusPwnens> and here is the pastebin with the error emssage and input
[Friday 03 September 2010] [13:35:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [13:40:34] <sustrik> ModusPwnens: throughput test?
[Friday 03 September 2010] [13:40:46] <sustrik> try increasing the sleep period in remote_the
[Friday 03 September 2010] [13:41:02] <sustrik> otherwise it doesn’t have enough time to send all the messages
[Friday 03 September 2010] [13:41:57] <ModusPwnens> Ok, I will try that sustrik
[Friday 03 September 2010] [13:42:08] <sustrik> cremes: nice!
[Friday 03 September 2010] [13:42:24] <sustrik> ha, that’s the guy complaining about the website!
[Friday 03 September 2010] [13:42:27] <sustrik> :)
[Friday 03 September 2010] [13:42:38] <cremes> it’s always good to see people writing articles on our favorite networking library
[Friday 03 September 2010] [13:43:23] <sustrik> yes, given how dull topic the networking is, it’s interesting that people even bother
[Friday 03 September 2010] [13:43:40] <cremes> heh
[Friday 03 September 2010] [13:43:42] <sustrik> btw, the site is back to normal
[Friday 03 September 2010] [13:45:51] <cremes> i see that; btw, i do like the new
*.com site
[Friday 03 September 2010] [13:45:54] <cremes> nice and simple
[Friday 03 September 2010] [13:46:02] Part mikejs has left this channel.
[Friday 03 September 2010] [13:46:19] <sustrik> few minor things to fix still...
[Friday 03 September 2010] [13:46:40] <sustrik> the link in the bottom look like there were 6 of them
[Friday 03 September 2010] [13:46:44] <sustrik> while there are only 3
[Friday 03 September 2010] [13:48:15] <sustrik> etc.
[Friday 03 September 2010] [14:06:46] <ModusPwnens> sustrik: I tried your suggestion and received a different error message
[Friday 03 September 2010] [14:07:21] <sustrik> what error message?
[Friday 03 September 2010] [14:07:51] <ModusPwnens> I just added it to the bottom
[Friday 03 September 2010] [14:08:04] <sustrik> well, not enough memory
[Friday 03 September 2010] [14:08:33] <ModusPwnens> so is that expected then?
[Friday 03 September 2010] [14:08:44] <sustrik> messages are held in memory
[Friday 03 September 2010] [14:09:02] <sustrik> so if you create more than your memory can hold, it’s expected
[Friday 03 September 2010] [14:09:11] <sustrik> what you can do though
[Friday 03 September 2010] [14:09:14] <ModusPwnens> Hmm, that makes sense.
[Friday 03 September 2010] [14:09:19] <sustrik> is to use ZMQ_SWAP socket option
[Friday 03 September 2010] [14:09:28] <sustrik> to offload excessive messages to the disk
[Friday 03 September 2010] [14:10:13] <ModusPwnens> Ok. I will try that.
[Friday 03 September 2010] [14:52:54] <lvh> hey
[Friday 03 September 2010] [14:53:11] <lvh> is there any progress or change regarding the zeromq-exposing-a-fd idea?
[Friday 03 September 2010] [14:53:24] <lvh> AFAIK this is what gevent and twisted are waiting for so we can integrate zeromq.
[Friday 03 September 2010] [14:53:43] <lvh> (I want to try zeromq, but if it won’t work with all of my existing event loop-based stuff, like twisted, well, I basically can’t use it.)
[Friday 03 September 2010] [14:54:39] <sustrik> lvh: it’s in trunk
[Friday 03 September 2010] [14:55:04] <sustrik> it’s a development version so it can be a bit unstable still
[Friday 03 September 2010] [14:55:25] <lvh> sustrik: oh cool, that’s interesting
[Friday 03 September 2010] [14:55:26] <sustrik> but it has ZMQ_FD socket option to get an underlying file descriptor from a 0MQ socket
[Friday 03 September 2010] [14:55:56] <sustrik> and ZMQ_EVENTS option that allows you to check whether socket is ready for readin and/or writing
[Friday 03 September 2010] [14:55:58] <lvh> git://github.com/zeromq/zeromq2.git ?
[Friday 03 September 2010] [14:56:04] <sustrik> yes
[Friday 03 September 2010] [14:56:07] <sustrik> “master”
[Friday 03 September 2010] [14:56:28] <lvh> Wait, why do people need to know the underlying fd
[Friday 03 September 2010] [14:56:46] <sustrik> to poll on it
[Friday 03 September 2010] [14:56:49] <lvh> I thought a zmq socket abstracted away from that and not necessarily have an fd, or not necessarily have exactly one
[Friday 03 September 2010] [14:57:00] <lvh> I’m not sure I understand. Isn’t that zmq’s job still?
[Friday 03 September 2010] [14:57:11] <sustrik> how would you poll on something that’s not a fd?
[Friday 03 September 2010] [14:57:16] <lvh> Is the idea to not use the IO parts in ZMQ?
[Friday 03 September 2010] [14:57:48] <lvh> sustrik: Oh, wait, sorry, I misunderstood what it does.
[Friday 03 September 2010] [14:57:51] <sustrik> the underlying fd is not the networking fd
[Friday 03 September 2010] [14:57:55] <lvh> Right.
[Friday 03 September 2010] [14:57:58] <sustrik> it’s 0MQ’s internal fd
[Friday 03 September 2010] [14:58:05] <lvh> Right, yes, that makes a lot more sense.
[Friday 03 September 2010] [14:58:05] <sustrik> used to communicate with I/O threads
[Friday 03 September 2010] [14:58:48] * sustrik is heading for pub
[Friday 03 September 2010] [15:00:03] <lvh> sustrik: have fun
[Friday 03 September 2010] [15:05:14] Quit lvh has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [15:08:21] Join mikejs has joined this channel (
~me@mikej.st).
[Friday 03 September 2010] [15:20:42] <bgranger> foo = Float(0.00001, config=True)
[Friday 03 September 2010] [15:20:56] Quit keffo has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [15:51:31] Quit GeekGod has left this server (Quit: GeekGod).
[Friday 03 September 2010] [16:32:08] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [16:48:25] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [16:49:07] Quit
mw__ has left this server (Changing host).
[Friday 03 September 2010] [17:34:54] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 03 September 2010] [17:41:26] Quit gebi has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [17:51:13] <troutwine> Does anyone have examples of Google’s Protocol Buffers being used as a serialization format with zeromq? I’ve seen it referenced but cannot find any examples.
[Friday 03 September 2010] [17:59:28] Quit keffo has left this server (Ping timeout: 272 seconds).
[Friday 03 September 2010] [18:06:40] Quit troutwine has left this server (Quit: leaving).
[Friday 03 September 2010] [18:14:11] <erickt> anyone who works on pyzmq online today? I’m having trouble trying to make a fedora rpm for the head of the tree
[Friday 03 September 2010] [18:15:15] <erickt> the commit 2a192b9 seems to have caused fedora 13’s rpmbuild to error out
[Friday 03 September 2010] [18:21:57] Quit sjampoo has left this server (Quit: zzz).
[Friday 03 September 2010] [18:22:32] Quit sjampoo has left this server (Client Quit).
[Friday 03 September 2010] [18:43:55] <bgranger> erickt: Hi I have a few minutes...
[Friday 03 September 2010] [18:44:25] <erickt> I think I found the problem, the enums for the devices is wrong
[Friday 03 September 2010] [18:44:30] <erickt> if I understand cython
[Friday 03 September 2010] [18:44:43] <bgranger> Possibly...
[Friday 03 September 2010] [18:45:11] <bgranger> What do you think the issue it.
[Friday 03 September 2010] [18:45:12] <erickt> ZMQ_QUEUE is defined to 1 in 2.0.8, ZMQ_FORWARDER 2, ZMQ_STREAMER to 3
[Friday 03 September 2010] [18:45:37] <erickt> well i haven’t completely confirmed it, but it seems to cause rpmbuild to fail on f13 I think
[Friday 03 September 2010] [18:45:44] <bgranger> Is that different than 2.0.7?
[Friday 03 September 2010] [18:45:58] Quit __moore__ has left this server (Quit: Leaving).
[Friday 03 September 2010] [18:46:16] <bgranger> Do you have an error msg or traceback?
[Friday 03 September 2010] [18:46:29] <Tasser> what’s a context exaclty?
[Friday 03 September 2010] [18:46:54] <erickt> bgranger: no, that’s part of the problem
[Friday 03 September 2010] [18:47:22] <erickt> it’s dying somewhere deep in a redhat command that doesn’t say really what’s going on
[Friday 03 September 2010] [18:47:31] <bgranger> Ahh. It is possible that the _zmq.c needs to be re-created for 2.0.8, but I would be a bit surprised.
[Friday 03 September 2010] [18:47:35] <bgranger> Can you build by hand?
[Friday 03 September 2010] [18:47:46] <erickt> i just know that in 1770dbb I can build an rpm, but with 2a192b96 i cannot
[Friday 03 September 2010] [18:47:47] <bgranger> That should show any issues outright
[Friday 03 September 2010] [18:47:57] <erickt> yeah it builds fine, and seems to be usable
[Friday 03 September 2010] [18:48:03] <bgranger> Are you buildling against 2.0.8 of zeromq
[Friday 03 September 2010] [18:48:05] <erickt> which is why I’m confused
[Friday 03 September 2010] [18:48:06] <erickt> yeah
[Friday 03 September 2010] [18:48:24] <bgranger> Hmmm, that is odd then. Do you have the script that rpmbuild is using?
[Friday 03 September 2010] [18:48:31] <erickt> Tasser: it’s an object to hold all the threads and underlying bits that zmq needs
[Friday 03 September 2010] [18:48:38] <ModusPwnens> troutwine, are you still here? I can help you!
[Friday 03 September 2010] [18:48:44] <bgranger> I do have to leave in a few minutes, so maybe post to the list if you can’t figure it out.
[Friday 03 September 2010] [18:49:15] <ModusPwnens> Also, I am receiving more errors with the benchmarking utility
[Friday 03 September 2010] [18:49:20] <ModusPwnens> this time not associated with large parameters
[Friday 03 September 2010] [18:49:22] <Tasser> erickt, can you have more than once context? not likely?
[Friday 03 September 2010] [18:49:34] <bgranger> Tasser: multiple contexts are just fine.
[Friday 03 September 2010] [18:49:53] <Tasser> hmm
[Friday 03 September 2010] [18:49:59] <bgranger> Tasser: the general guideline is 1 per library. You can use it to isolate your own zmq stuff from that of other libs.
[Friday 03 September 2010] [18:50:00] <Tasser> I’ll have to play around a bit
[Friday 03 September 2010] [18:50:08] <erickt> that’s the rpm spec and a driver script
[Friday 03 September 2010] [18:51:55] <bgranger> Do you know how far it gets?
[Friday 03 September 2010] [18:52:21] <erickt> it compiles and it’s just determining metadata for the rpm
[Friday 03 September 2010] [18:52:24] <bgranger> erickt: sorry I need to run now, can you email the list. Also, have you talked to T. Spura about this?
[Friday 03 September 2010] [18:52:32] <erickt> nope, just found this
[Friday 03 September 2010] [18:57:32] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [19:02:32] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [19:08:28] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [19:09:07] Quit
mw__ has left this server (Changing host).
[Friday 03 September 2010] [19:40:50] Quit
feroz_ has left this server (Ping timeout: 264 seconds).
[Friday 03 September 2010] [19:41:34] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Friday 03 September 2010] [19:43:15] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [19:57:13] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [19:57:45] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [20:14:13] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [21:18:32] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [21:19:30] <lestrrat> sustrik: I’m going to be out for the day, so won’t be able to check for another 8hrs or so
[Friday 03 September 2010] [21:19:41] <lestrrat> but will do when I get back
[Friday 03 September 2010] [21:20:28] Nick
AndrewBC_ is now known as AndrewBC.
[Friday 03 September 2010] [21:22:18] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [21:46:28] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [22:09:41] Quit jsimmons has left this server (Ping timeout: 255 seconds).
[Friday 03 September 2010] [22:46:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [23:36:02] Quit bgranger has left this server (Quit: bgranger).
[Friday 03 September 2010] [23:43:39] Quit bgranger has left this server (Client Quit).
[Saturday 04 September 2010] [00:03:29] Quit keffo has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 4 01:30:09 2010
[Saturday 04 September 2010] [01:30:09] Topic The channel topic is “Welcome!”.
[Saturday 04 September 2010] [01:30:15] Mode Channel modes: no messages from outside, topic protection
[Saturday 04 September 2010] [01:30:16] Created This channel was created on 2010-02-05 10:44.
[Saturday 04 September 2010] [02:01:47] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [02:03:12] Join joevandyk has joined this channel (
~joe@98.117.99.141).
[Saturday 04 September 2010] [02:03:38] <joevandyk> hello!
[Saturday 04 September 2010] [02:04:26] <joevandyk> i’m using the ruby zmq gem. i’m using the req/res sockets. When i try to send a message after a client has disconnected and i’ve specified no blocking, the sender just blocks forever.
[Saturday 04 September 2010] [02:13:27] <joevandyk> I think I see the problem. When the service disconnects, the client still tries to send a message, and is waiting for a response.
[Saturday 04 September 2010] [02:13:34] <joevandyk> is there a way around that?
[Saturday 04 September 2010] [02:14:02] <joevandyk> i wonder if this is a bug in the ruby library
[Saturday 04 September 2010] [02:14:37] <lestrrat> sustrik: I sent a message to the mailing list, but I messed my From: field :/
[Saturday 04 September 2010] [02:15:11] <lestrrat> sustrik: anyway, I all I wanted to say was that the eintr branch worked :)
[Saturday 04 September 2010] [02:40:24] Quit bgranger has left this server (Quit: bgranger).
[Saturday 04 September 2010] [02:43:59] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [03:25:08] Quit keffo has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [03:39:46] Quit guido_g has left this server (Quit: Boom...).
[Saturday 04 September 2010] [03:51:24] <sustrik> lestrrat: great, thanks!
[Saturday 04 September 2010] [03:51:44] <sustrik> joevandyk: it’s the way 0mq works
[Saturday 04 September 2010] [03:52:26] <joevandyk> seems odd that if the service disconnects that the client is hung forever
[Saturday 04 September 2010] [03:52:55] <sustrik> it’ll continue working once service becaomes available
[Saturday 04 September 2010] [03:53:34] <sustrik> if you want to explicitly exit in case your task haven’t been completed in a predefined time, use timeouts
[Saturday 04 September 2010] [03:59:04] <joevandyk> sustrik: hm, i don’t see how it can become unblocked once it’s waiting for a recv
[Saturday 04 September 2010] [03:59:19] <sustrik> use zmq_poll
[Saturday 04 September 2010] [03:59:24] <sustrik> there’s a timout parameter there
[Saturday 04 September 2010] [03:59:41] <joevandyk> sustrik: what i’m not understanding is why the client sends a message when there’s no servers to receive it
[Saturday 04 September 2010] [04:00:13] <sustrik> what does “there’s no server” mean?
[Saturday 04 September 2010] [04:00:43] <sustrik> there’s no such thing in networking
[Saturday 04 September 2010] [04:01:10] <sustrik> what you have otoh is “the transfer haven’t been acknowledged in time”
[Saturday 04 September 2010] [04:01:29] <sustrik> which is the timeout
[Saturday 04 September 2010] [04:02:47] <joevandyk> sorry – when the REP socket disconnects. The REQ client had been happily sending data to the REP. But then the REP disconnected. The REQ tried to send data again to the REP and the send suceeded, but there was nothing to send it to.
[Saturday 04 September 2010] [04:19:08] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 04 September 2010] [04:37:29] Quit joevandyk has left this server (Quit: joevandyk).
[Saturday 04 September 2010] [06:09:07] Quit keffo has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [06:19:40] Quit keffo has left this server (Ping timeout: 258 seconds).
[Saturday 04 September 2010] [06:20:50] Join stephank has joined this channel (
~shteef@84.25.113.14).
[Saturday 04 September 2010] [06:22:11] Nick xla is now known as xla_soundcloud.
[Saturday 04 September 2010] [07:29:29] Nick xla_soundcloud is now known as xla_SC.
[Saturday 04 September 2010] [07:40:58] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [08:14:19] <Tasser> cremes, got me the big picture for zmqmachine?
[Saturday 04 September 2010] [08:24:33] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 04 September 2010] [08:33:10] <mato> re
[Saturday 04 September 2010] [08:33:44] <sustrik> hi
[Saturday 04 September 2010] [08:34:11] <mato> sustrik: list of patches to cherry-pick along with instructions coming up shortly
[Saturday 04 September 2010] [08:34:38] <sustrik> thx
[Saturday 04 September 2010] [08:34:40] <mato> sustrik: just need to do some housekeeping (also delete and recreate FD_SETSIZE issue in tracker since I found that bug, not pieter)
[Saturday 04 September 2010] [08:34:54] <sustrik> ok
[Saturday 04 September 2010] [08:42:42] <mato> ok, i can’t delete it, have commented with the current situation and removed the “maint” label
[Saturday 04 September 2010] [08:44:09] <sustrik> fine
[Saturday 04 September 2010] [08:49:44] <mato> sustrik: mumble... pieter also deleted the “empty” zmq_forwarder/streamer/queue documentation
[Saturday 04 September 2010] [08:49:57] <mato> sustrik: this has packaging implications
[Saturday 04 September 2010] [08:50:05] <mato> sustrik: how would you like me to resolve this?
[Saturday 04 September 2010] [08:50:10] <sustrik> get it back
[Saturday 04 September 2010] [08:50:31] <mato> ok, what about the zmq_device documentation he added?
[Saturday 04 September 2010] [08:50:46] <mato> are devices still experimental in 2.0.x?
[Saturday 04 September 2010] [08:51:34] <mato> or would you prefer to revert his commit entirely?
[Saturday 04 September 2010] [08:51:45] <mato> the problem is it has other stuff mixed up in it
[Saturday 04 September 2010] [08:51:59] <mato> like changes to asciidoc.conf which are half-broken
[Saturday 04 September 2010] [08:52:10] <sustrik> is there any problem with device documentation
[Saturday 04 September 2010] [08:52:29] <mato> i don’t know, but you were keeping it as undocumented
[Saturday 04 September 2010] [08:52:40] <mato> there is a problem with the other parts of the commit
[Saturday 04 September 2010] [08:53:22] <mato> like the asciidoc footer only being present for manpages, not for html
[Saturday 04 September 2010] [08:53:28] <sustrik> what’s easier for you?
[Saturday 04 September 2010] [08:54:22] <mato> less work right now would be just to revert the whole commit
[Saturday 04 September 2010] [08:54:49] <mato> and pieter can resubmit his changes
[Saturday 04 September 2010] [08:55:14] <mato> hmm, maybe
[Saturday 04 September 2010] [08:55:45] <sustrik> it’s up to you
[Saturday 04 September 2010] [09:15:30] Quit xla_SC has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [09:25:42] Quit rgl has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [10:03:23] <CIA-20> zeromq2: 03Martin Lucina 07master * r2673a84 10/ (34 files): (log message trimmed)
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: doc: Update zmq_socket(3) for 2.0.8 API changes
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Added man page for the zmq_device method”
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Added clean target that deletes generated man pages”
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Various changes to documentation project:”
[Saturday 04 September 2010] [10:03:34] <CIA-20> zeromq2: 03Martin Lucina 07maint * ree3444f 10/ doc/zmq_socket.txt : doc: Update zmq_socket(3) for 2.0.8 API changes (+8 more commits...) -
http://bit.ly/aRdJ8z[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: 03Martin Lucina 07master * rf850190 10/ include/zmq.h : zmq.h: Fix typo and use of C99 comment -
http://bit.ly/95mVlM[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: 03Martin Lucina 07master * r51a84c1 10/ (src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: Ensure that 0MQ does not attempt to call select() on more than FD_SETSIZE
[Saturday 04 September 2010] [10:25:09] <CIA-20> zeromq2: 03Martin Lucina 07master * rca17612 10/ (include/zmq.h src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:10] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [10:25:10] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [10:25:11] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:11] <CIA-20> zeromq2: zmq.h: Fix typo and use of C99 comment
[Saturday 04 September 2010] [10:25:12] <CIA-20> zeromq2: Conflicts:
[Saturday 04 September 2010] [10:25:15] <CIA-20> zeromq2: 03Martin Lucina 07maint * rf850190 10/ include/zmq.h : zmq.h: Fix typo and use of C99 comment -
http://bit.ly/95mVlM[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: 03Martin Lucina 07maint * r51a84c1 10/ (src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: Ensure that 0MQ does not attempt to call select() on more than FD_SETSIZE
[Saturday 04 September 2010] [10:43:08] Quit plq has left this server (Remote host closed the connection).
[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Ivo Danihelka 07maint * rae567be 10/ (AUTHORS src/zmq.cpp): improved null checking in zmq_term -
http://bit.ly/anJS6s[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Jon Dyte 07maint * rc2f3b3b 10/ (src/forwarder.cpp src/queue.cpp src/streamer.cpp): forwarder and streamer devices handle multi-part messages correctly -
http://bit.ly/aRxGoy[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Dhammika Pathirana 07maint * r1022789 10/ src/zmq_decoder.cpp : assert on malformed messages -
http://bit.ly/907fWH[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rdb7fe85 10/ include/zmq.h : Broken device numbering reverted -
http://bit.ly/cncpxn[Saturday 04 September 2010] [11:19:45] Quit poswald has left this server (Quit: poswald).
[Saturday 04 September 2010] [11:20:13] <CIA-20> zeromq2: 03Ivo Danihelka 07master * rae567be 10/ (AUTHORS src/zmq.cpp): improved null checking in zmq_term -
http://bit.ly/anJS6s[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Jon Dyte 07master * rc2f3b3b 10/ (src/forwarder.cpp src/queue.cpp src/streamer.cpp): forwarder and streamer devices handle multi-part messages correctly -
http://bit.ly/aRxGoy[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Dhammika Pathirana 07master * r1022789 10/ src/zmq_decoder.cpp : assert on malformed messages -
http://bit.ly/907fWH[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Martin Sustrik 07master * rdb7fe85 10/ include/zmq.h : Broken device numbering reverted -
http://bit.ly/cncpxn[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: 03Martin Sustrik 07master * r76f2e5d 10/ include/zmq.h : (log message trimmed)
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: Broken device numbering reverted
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: assert on malformed messages
[Saturday 04 September 2010] [11:20:25] <CIA-20> zeromq2: forwarder and streamer devices handle multi-part messages correctly
[Saturday 04 September 2010] [11:20:26] <CIA-20> zeromq2: improved null checking in zmq_term
[Saturday 04 September 2010] [11:22:45] Quit xla_SC has left this server (Ping timeout: 260 seconds).
[Saturday 04 September 2010] [11:34:39] <AndrewBC> and could use a fresh pair of eyes.
[Saturday 04 September 2010] [11:36:41] <AndrewBC> I also don’t see anything pertaining to this in the buglist
[Saturday 04 September 2010] [11:46:55] <AndrewBC> That gibberish output is looking more and more sinister, you know
[Saturday 04 September 2010] [11:47:00] <AndrewBC> I wonder what that’s about
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: 03Martin Lucina 07master * ra6d3629 10/ (Makefile.am configure.in ChangeLog): (log message trimmed)
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: build: Generate ChangeLog in ‘make dist’, ZIP automatically
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: Change ‘make dist’ to generate the Git ChangeLog file, that way it doesn’t
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: have to be manually updated nor kept in Git which causes unnecessary work.
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: Also change ‘make dist’ to invoke ‘dist-zip’ automatically to generate a
[Saturday 04 September 2010] [11:50:22] <CIA-20> zeromq2: ZIP as well as a .tar.gz.
[Saturday 04 September 2010] [11:50:23] <CIA-20> zeromq2: 03Martin Lucina 07master * r32fd916 10/ doc/asciidoc.conf :
[Saturday 04 September 2010] [11:50:24] <CIA-20> zeromq2: doc: Add 0MQ version to XHTML11 backend footer
[Saturday 04 September 2010] [11:50:24] <CIA-20> zeromq2: Thanks to Matt Weinstein for the suggestion. -
http://bit.ly/d1r24H[Saturday 04 September 2010] [11:50:25] <CIA-20> zeromq2: 03Martin Lucina 07master * r1e84519 10/ .gitignore : Update .gitignore -
http://bit.ly/bH7Jhp[Saturday 04 September 2010] [11:50:25] <CIA-20> zeromq2: 03Martin Lucina 07master * rd4c8de5 10/ (5 files in 2 dirs):
[Saturday 04 September 2010] [11:50:26] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [11:50:26] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [11:50:27] <CIA-20> zeromq2: Update .gitignore
[Saturday 04 September 2010] [11:50:27] <CIA-20> zeromq2: doc: Add 0MQ version to XHTML11 backend footer
[Saturday 04 September 2010] [11:54:08] <AndrewBC> Okay, I tracked it down to the zmq_recv() in my client. Perhaps there’s something special in the flags I need to be aware of since it’s on the other side of an XREP
[Saturday 04 September 2010] [11:54:30] Quit gebi has left this server (Read error: Operation timed out).
[Saturday 04 September 2010] [11:54:48] <AndrewBC> So I’m reading the API docs now
[Saturday 04 September 2010] [12:01:34] <AndrewBC> beginning to suspect that I should be checking for multipart messages
[Saturday 04 September 2010] [12:06:26] <sccolbert> Perhaps a simple problem, but I appear to have a mem leak on the SUB side of a PUB/SUB pattern
[Saturday 04 September 2010] [12:06:50] <sccolbert> when I connect the SUB to the PUB, any messages not matching the filter cause wasted memory
[Saturday 04 September 2010] [12:07:30] <sccolbert> a simple test send say, 1 million short messages that dont match any filter on the SUB, will cause the memory usage of the SUB to explode
[Saturday 04 September 2010] [12:07:55] <sccolbert> has anyone else experienced this?
[Saturday 04 September 2010] [12:08:47] <AndrewBC> I assume that since the filtering is done on the SUB side, the messages are indeed being transmitted to the socket on that end
[Saturday 04 September 2010] [12:09:50] <sccolbert> this is true, but their memory is never freed
[Saturday 04 September 2010] [12:10:13] <AndrewBC> Are you using a binding, or what?
[Saturday 04 September 2010] [12:10:22] <sccolbert> PyZMQ
[Saturday 04 September 2010] [12:10:28] <sccolbert> on ubuntu 10.04
[Saturday 04 September 2010] [12:11:00] <sccolbert> so if I just connect the SUB and dont set any filter, the SUB process quickly locks up the system
[Saturday 04 September 2010] [12:11:30] <sccolbert> which I cant see as being related to PyZMQ
[Saturday 04 September 2010] [12:12:15] <AndrewBC> I’m not sure, since I’m not a dev of zmq. It’s supposed to handle the memory just fine, if wishes were horses...
[Saturday 04 September 2010] [12:12:45] <AndrewBC> It could be either, though. All it takes is a circular reference for python to not garbage collect objects
[Saturday 04 September 2010] [12:13:11] <AndrewBC> circular, cyclic, bah
[Saturday 04 September 2010] [12:14:00] <sccolbert> this is true, though I would think that since I havent actually called recv() on the sub (since its filtering all messages) that python isnt even aware a message arrived
[Saturday 04 September 2010] [12:20:40] Quit xla_SC has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [12:21:46] <AndrewBC> Wish I could help you. I’m banging my head up against a bizzare failed assertion myself
[Saturday 04 September 2010] [12:34:40] Quit xla_SC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [12:38:37] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rebf3089 10/ NEWS : NEWS updated for 2.0.9 -
http://bit.ly/doVZJ4[Saturday 04 September 2010] [12:45:32] Quit AndrewBC has left this server (Ping timeout: 272 seconds).
[Saturday 04 September 2010] [12:56:38] <pieter_hintjens> sccolbert: what version of 0MQ are you using?
[Saturday 04 September 2010] [12:58:09] <sccolbert> pieter_hintjens: 2.0.7
[Saturday 04 September 2010] [12:58:50] <pieter_hintjens> when you stop the publisher does the memory consumption of the subscriber go down again?
[Saturday 04 September 2010] [12:59:03] <sccolbert> hmm havent tried that
[Saturday 04 September 2010] [12:59:05] <sccolbert> gimme a sec
[Saturday 04 September 2010] [13:01:36] <sccolbert> no
[Saturday 04 September 2010] [13:01:50] <sccolbert> killing the publisher does not free the memory in the SUB
[Saturday 04 September 2010] [13:04:10] <pieter_hintjens> hmm... so the next steps are to test with the latest 2.0.9 release
[Saturday 04 September 2010] [13:04:19] <pieter_hintjens> and then make a minimal test case that reproduces this, and log an issue
[Saturday 04 September 2010] [13:04:30] <sccolbert> k, lemme grab the new one along with the new PyZMQ
[Saturday 04 September 2010] [13:04:36] <sccolbert> will only take me a couple minutes
[Saturday 04 September 2010] [13:04:52] <sccolbert> a different question:
[Saturday 04 September 2010] [13:04:54] <pieter_hintjens> if you can make a parallel test case in C or C++ that will help identify whether it’s in PyZMQ or 0MQ
[Saturday 04 September 2010] [13:05:04] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [13:05:04] <sccolbert> sure
[Saturday 04 September 2010] [13:05:14] <pieter_hintjens> shoot...
[Saturday 04 September 2010] [13:05:45] <sccolbert> in a pub/sub pattern, if i send a multipart message, and the first part doesnt match the sub filter, does the sub still recieve the rest of the message?
[Saturday 04 September 2010] [13:06:00] <sccolbert> and then just dump it
[Saturday 04 September 2010] [13:06:06] <pieter_hintjens> i actually added this to the guide today but did not publish it yet
[Saturday 04 September 2010] [13:06:14] <pieter_hintjens> a multipart message is one message
[Saturday 04 September 2010] [13:06:23] <pieter_hintjens> think of the first part as the envelope
[Saturday 04 September 2010] [13:06:32] <pieter_hintjens> it’s very convenient to put the pubsub key there
[Saturday 04 September 2010] [13:06:47] <pieter_hintjens> if the key does not match any subscription the entire message is discarded / ignored
[Saturday 04 September 2010] [13:07:11] <sccolbert> is there any performance improvement to be had with that, over just joining my key to the beggininng of my message?
[Saturday 04 September 2010] [13:07:29] <pieter_hintjens> using a pubsub envelope like this means you can do zerocopy on the data part
[Saturday 04 September 2010] [13:07:41] <sccolbert> ah, great
[Saturday 04 September 2010] [13:07:50] <pieter_hintjens> there is no significant performance cost and you can save on formatting the full message
[Saturday 04 September 2010] [13:08:23] <pieter_hintjens> plus you are guaranteed the key remains separate from the data
[Saturday 04 September 2010] [13:08:29] <pieter_hintjens> it’s a Good Thing
[Saturday 04 September 2010] [13:08:40] <sccolbert> yep
[Saturday 04 September 2010] [13:09:01] <sccolbert> also, in a general pub/sub pattern do you recommend epgm over tcp?
[Saturday 04 September 2010] [13:09:13] <CIA-20> zeromq2: 03Martin Sustrik 07maint * r01c463c 10/ (builds/msvc/platform.hpp configure.in): Version number incremented to 2.0.10 -
http://bit.ly/9vL8xS[Saturday 04 September 2010] [13:09:16] <sccolbert> I wasnt able to find much literature on the pro’s vs cons of each
[Saturday 04 September 2010] [13:10:11] <pieter_hintjens> you’d only use pgm/epgm if you have a performance issue with tcp
[Saturday 04 September 2010] [13:10:23] <pieter_hintjens> there is ongoing work to make tcp pubsub much more efficient for large fanouts
[Saturday 04 September 2010] [13:10:37] <pieter_hintjens> in general use tcp unless you know why you need multicast
[Saturday 04 September 2010] [13:10:38] <sccolbert> cool
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rebf3089 10/ NEWS : NEWS updated for 2.0.9 -
http://bit.ly/doVZJ4[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r01c463c 10/ (builds/msvc/platform.hpp configure.in): Version number incremented to 2.0.10 -
http://bit.ly/9vL8xS[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rb4740c1 10/ NEWS : (log message trimmed)
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [13:10:48] <CIA-20> zeromq2: Version number incremented to 2.0.10
[Saturday 04 September 2010] [13:10:49] <CIA-20> zeromq2: NEWS updated for 2.0.9
[Saturday 04 September 2010] [13:10:49] <CIA-20> zeromq2: Conflicts:
[Saturday 04 September 2010] [13:10:50] <CIA-20> zeromq2: builds/msvc/platform.hpp
[Saturday 04 September 2010] [13:11:24] <Steve-o> usually tcp up to 4-5 peers is better than multicast for almost all scenarios
[Saturday 04 September 2010] [13:11:35] <sccolbert> good to know
[Saturday 04 September 2010] [13:11:40] <sccolbert> thanks
[Saturday 04 September 2010] [13:13:01] <Steve-o> usually NICs give you some form of TCP acceleration compared to none for UDP multicast
[Saturday 04 September 2010] [13:16:07] <sustrik> uh, 2.0.9 is out
[Saturday 04 September 2010] [13:16:38] * sustrik has enough, goes away to play moonlight sonata
[Saturday 04 September 2010] [13:47:53] <sccolbert> pieter_hintjens: tested new version in C and Python
[Saturday 04 September 2010] [13:47:57] <sccolbert> same issues
[Saturday 04 September 2010] [13:48:37] <sccolbert> where should I log the issue?
[Saturday 04 September 2010] [13:50:29] <sccolbert> ah, ok, it only happens in C when the sub hasnt set any filter
[Saturday 04 September 2010] [13:50:44] <sccolbert> setting a filter that doesnt match anything from the PUB solves the issue
[Saturday 04 September 2010] [13:50:50] <sccolbert> in C at least
[Saturday 04 September 2010] [13:51:23] <sccolbert> i will file this with Brian
[Saturday 04 September 2010] [14:18:12] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 04 September 2010] [14:25:46] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [14:41:13] Nick
AndrewBC_ is now known as AndrewBC.
[Saturday 04 September 2010] [15:01:00] <twcc> Hey all, I’m trying to figure out if zeromq ist the right tool for me. I want to have one “Server” and multiple client, which pushes messages to the server. I can’t find a pattern which fits really good - Request-reply would fit best, but as I understood I have to send a reply. - Is this case so trivial, that I should fall back to standard socket programming?
[Saturday 04 September 2010] [15:11:41] Quit xla_SC has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [15:37:54] <Steve-o> twcc: sounds like PUB/SUB
[Saturday 04 September 2010] [15:43:36] <twcc> Because of the unidirectional manner this would fit, but AFAIK the Publisher starts the socket and the subscriber has to connect?! - This could run into problems with a changing number of publisher, or can I connect them to one socket, and zmq handles this?
[Saturday 04 September 2010] [15:53:17] <Steve-o> You can connect multiple to one zmq socket as detailed in zmq_socket(3)
[Saturday 04 September 2010] [15:54:40] <Steve-o> there is currently no bus topology socket pattern
[Saturday 04 September 2010] [15:55:13] <Steve-o> you can jump down to raw PGM for that
[Saturday 04 September 2010] [15:57:09] <twcc> maybe i’m stuck to much to socket programming ... - so I can bind multiple publisher to the same socket/port and have one subscriber which processes the messages? That would be great and solves all my problems :-)
[Saturday 04 September 2010] [16:01:35] <kooroo> hey folks, I’m having trouble compiling zeromq on rh5 x86_64
[Saturday 04 September 2010] [16:01:52] <kooroo> .libs/libzmq_la-txwi.o: relocation R_X86_64_PC32 against
`pgm_rs_create’ can not be used when making a shared object; recompile with -fPIC is what I’m getting.
[Saturday 04 September 2010] [16:02:00] <kooroo> has anyone come across this before?
[Saturday 04 September 2010] [16:02:11] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [16:03:06] <Steve-o> twcc: for each subscriber you call zmq_connect for each publisher on one zmq socket
[Saturday 04 September 2010] [16:04:49] <Steve-o> kooroo: it is a bug in RHEL GCC, you can disable PGM or there is a flag you can set to disable visibility on the PGM internal API
[Saturday 04 September 2010] [16:07:08] <kooroo> Steve-o, is this the -DPGM_GNUC_INTERNAL flag?
[Saturday 04 September 2010] [16:07:29] <Steve-o> kooroo: yes, it needs to be “-DPGM_GNUC_INTERNAL=” for broken GCC, default “-DPGM_GNUC_INTERNAL=G_GNUC_INTERNAL”
[Saturday 04 September 2010] [16:08:36] <Steve-o> I think someone found a backported GCC version that works too on the list before
[Saturday 04 September 2010] [16:10:45] <kooroo> Steve-o: do you know offhand if upgrading gcc works?
[Saturday 04 September 2010] [16:11:07] <Steve-o> 4.1.2 is the affected version
[Saturday 04 September 2010] [16:12:36] <kooroo> Steve-o: thanks, Imma upgrade gcc and try it again.
[Saturday 04 September 2010] [16:12:54] <twcc> Hm, maybe I explained it bad - I have a changing number of processes (publisher) which have to send messages to one sink (daemon/subscriber), which handles these messages. A topologie where the daemon/subscriber has to connect to the publisher isn’t feasible. - So the client/server application would fit best, except that the server has to response to a message.
[Saturday 04 September 2010] [16:13:50] <kooroo> twcc, could you have them all subscribe to the same multicast stream and only the subscriber react to messages?
[Saturday 04 September 2010] [16:14:15] <kooroo> not the most efficient way, I admit.
[Saturday 04 September 2010] [16:17:35] <twcc> hm, funny idea, didn’t thought of it
[Saturday 04 September 2010] [16:19:05] <twcc> multicast is only possible on network connections?, because all the processes are running on the same computer - so i would like to use ipc
[Saturday 04 September 2010] [16:24:02] <Steve-o> multicast requires a network for reliable delivery, it’s pretty useless for ipc.
[Saturday 04 September 2010] [16:30:35] <Steve-o> I think as IPC is currently implemented using local sockets you would have to add a shared memory transport for localhost broadcast
[Saturday 04 September 2010] [16:32:58] <twcc> I think the multicast idea would only be a workaround - I would like to have a non-blocking “Request-reply” without reply pattern, but I can’t figure out, how to realize it with zeromq
[Saturday 04 September 2010] [16:36:14] <twcc> Thank you for your help - I will try if the request-reply pattern satisfy the requirements and handle the non-blocking by myself
[Saturday 04 September 2010] [16:43:48] <Steve-o> post a message on the list too, it sounds like a good idea for v3, I think 29West have a broadcast IPC transport too
[Saturday 04 September 2010] [16:50:21] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [16:54:11] Part twcc has left this channel.
[Saturday 04 September 2010] [16:58:57] Quit xla_SC has left this server (Quit: leaving).
[Saturday 04 September 2010] [17:55:35] Quit sccolbert has left this server (Remote host closed the connection).
[Saturday 04 September 2010] [18:23:00] Quit Odizes has left this server (Quit: Saliendo).
[Saturday 04 September 2010] [18:37:19] <DasIch> (obviously what i really did was a bit more complex but that what it boils down to)
[Saturday 04 September 2010] [18:40:56] <lestrrat> sustrik: so the eintr thins is going to be in 2.1?
[Saturday 04 September 2010] [19:01:44] <lestrrat> sustrik: And the ZMQ_FD stuff to get at the file descriptor – is that going to be in 2.1 as well?
[Saturday 04 September 2010] [19:07:22] Quit bgranger has left this server (Quit: bgranger).
[Saturday 04 September 2010] [19:13:20] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Saturday 04 September 2010] [20:01:46] Quit Guest26371 has left this server (Quit: Page closed).
[Saturday 04 September 2010] [20:15:23] Quit DasIch has left this server (Ping timeout: 255 seconds).
[Saturday 04 September 2010] [21:13:32] Quit andrewvc has left this server (Quit: andrewvc).
[Saturday 04 September 2010] [21:45:06] Quit andrewvc has left this server (Quit: andrewvc).
[Saturday 04 September 2010] [21:49:55] Quit xla has left this server (Quit: leaving).
[Saturday 04 September 2010] [22:12:10] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [22:34:34] Quit plq has left this server (Quit: Leaving).
[Saturday 04 September 2010] [22:43:18] Quit jonrafkind has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [23:15:29] Quit sd88g93 has left this server (Quit: Ex-Chat).
[Saturday 04 September 2010] [23:21:28] Quit jonrafkind has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [01:40:12] <sustrik> lestrrat: both are going to be in 2.1
[Sunday 05 September 2010] [01:40:45] <sustrik> ZMQ_FD as already there
[Sunday 05 September 2010] [01:41:00] <sustrik> i’ll write an email about EINTR & 2.1 today
[Sunday 05 September 2010] [02:00:28] <sustrik> DasIch: That’s a bug in OpenPGM library
[Sunday 05 September 2010] [02:00:53] <sustrik> if you don’t need PGM, just compile 0MQ without it
[Sunday 05 September 2010] [02:01:36] <sustrik> it have been fixed in newer OpenPGM releases
[Sunday 05 September 2010] [02:01:44] <sustrik> but haven’t got into 0MQ as for now
[Sunday 05 September 2010] [02:01:59] <sustrik> anway, creating multiple contexts is pretty useless
[Sunday 05 September 2010] [02:02:08] <sustrik> why do you do that?
[Sunday 05 September 2010] [02:32:34] <lestrrat> sustrik: by “it’s already there” you mean the github repo, right?
[Sunday 05 September 2010] [02:34:06] <sustrik> yes
[Sunday 05 September 2010] [02:34:09] <sustrik> the master branch
[Sunday 05 September 2010] [02:34:33] <sustrik> EINTR on eintr branch, FD & EVENTS on master branch
[Sunday 05 September 2010] [02:35:21] <sustrik> if i have some spare time today, i’ll merge EINTR fix to master as well
[Sunday 05 September 2010] [02:57:14] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 05 September 2010] [03:29:53] Nick
AndrewBC_ is now known as AndrewBC.
[Sunday 05 September 2010] [03:57:15] Topic The channel topic is “Welcome!”.
[Sunday 05 September 2010] [03:57:21] Mode Channel modes: no messages from outside, topic protection
[Sunday 05 September 2010] [03:57:21] Created This channel was created on 2010-02-05 10:44.
[Sunday 05 September 2010] [03:57:29] Quit travlr has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [04:17:47] Quit kooroo has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [04:17:47] Nick kooroox2 is now known as kooroo.
[Sunday 05 September 2010] [04:39:40] <pieter_hintjens> sustrik: hi
[Sunday 05 September 2010] [05:50:29] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 05 September 2010] [05:54:31] <sustrik> pieter_hintjens: hi
[Sunday 05 September 2010] [05:57:29] <pieter_hintjens> hi martin... i was wondering about different ways to submit patches
[Sunday 05 September 2010] [05:57:47] <pieter_hintjens> people are using at least three different routes
[Sunday 05 September 2010] [05:57:59] <pieter_hintjens> patches to zeromq-dev, issues, and pull requests
[Sunday 05 September 2010] [05:58:48] <sustrik> i am using patches from the mailing list
[Sunday 05 September 2010] [05:58:56] <sustrik> are there any pending patches anywhere else?
[Sunday 05 September 2010] [05:59:05] <pieter_hintjens> i saw one pull request on the ML today
[Sunday 05 September 2010] [05:59:22] <sustrik> yup
[Sunday 05 September 2010] [05:59:35] <sustrik> that’s a packaging script thing, it’s up to mato to manage that one
[Sunday 05 September 2010] [05:59:47] * sustrik has no idea how packaging works
[Sunday 05 September 2010] [06:00:04] <pieter_hintjens> :-) it’s more of a consistency thing than anything else
[Sunday 05 September 2010] [06:00:27] <pieter_hintjens> ok, totally different thing, i have a trivial inproc example that refuses to work
[Sunday 05 September 2010] [06:00:39] <sustrik> can you paste is somewhere?
[Sunday 05 September 2010] [06:00:51] <pieter_hintjens> yes, i raised an issue (62) with an example and analysis
[Sunday 05 September 2010] [06:01:00] <sustrik> let me see
[Sunday 05 September 2010] [06:01:56] <pieter_hintjens> i’ve spent several hours trying different things, ended up doing a sleep... :-/
[Sunday 05 September 2010] [06:03:27] <sustrik> i would say it’s the inproc reconnect thing
[Sunday 05 September 2010] [06:03:40] <sustrik> inproc reconnect is still unimplemented
[Sunday 05 September 2010] [06:03:58] <sustrik> thus doing connect before bind does not work
[Sunday 05 September 2010] [06:04:34] <sustrik> so, if step1 connects before step2 binds
[Sunday 05 September 2010] [06:04:38] <sustrik> :|
[Sunday 05 September 2010] [06:04:39] <pieter_hintjens> ah, it’s not just for PAIR sockets
[Sunday 05 September 2010] [06:05:08] <sustrik> it’s inproc as well
[Sunday 05 September 2010] [06:05:18] <pieter_hintjens> with PUSH/PULL it works a little better, say 90% of the time
[Sunday 05 September 2010] [06:05:19] <sustrik> it’s not that visible as inproc connection cannot break
[Sunday 05 September 2010] [06:05:25] <pieter_hintjens> with PAIR it dies almost immediately
[Sunday 05 September 2010] [06:05:35] <sustrik> it’s a timing issue
[Sunday 05 September 2010] [06:05:44] <pieter_hintjens> ok, if this is a known restriction that’s fine, I can document it
[Sunday 05 September 2010] [06:06:19] <sustrik> Connecting a socket
[Sunday 05 September 2010] [06:06:19] <sustrik> When connecting a socket to a peer address using zmq_connect() with the inproc transport, the endpoint shall be interpreted as an arbitrary string identifying the name to connect to. The name
[Sunday 05 September 2010] [06:06:19] <sustrik> must have been previously created by assigning it to at least one socket within the same 0MQ context as the socket being connected.
[Sunday 05 September 2010] [06:06:19] <pieter_hintjens> there is no technical reason it could not work, right?
[Sunday 05 September 2010] [06:06:25] <sustrik> no
[Sunday 05 September 2010] [06:06:26] <pieter_hintjens> hmm, ok
[Sunday 05 September 2010] [06:06:32] <sustrik> just not implemented
[Sunday 05 September 2010] [06:06:39] <pieter_hintjens> excellent, that is all I need
[Sunday 05 September 2010] [06:06:44] <pieter_hintjens> i’ll close the issue
[Sunday 05 September 2010] [06:06:47] <pieter_hintjens> thx
[Sunday 05 September 2010] [06:06:50] <sustrik> np
[Sunday 05 September 2010] [06:08:10] <pieter_hintjens> “ZeroMQ is quickly becoming an even bigger hammer in the premature optimization planet of Newbo-Thumbia.”
[Sunday 05 September 2010] [06:08:24] <pieter_hintjens> vs. “Then they’d better find a new rite of passage, because ZeroMQ is turning into a category killer.”
[Sunday 05 September 2010] [06:08:36] <sustrik> :)
[Sunday 05 September 2010] [06:08:53] <pieter_hintjens> that is an excellent discussion...
[Sunday 05 September 2010] [06:09:00] <sustrik> yes, i liked it
[Sunday 05 September 2010] [06:09:20] <pieter_hintjens> i’m wondering if it’s worth resurrecting the old amqp file transfer semantics as an example over 0MQ
[Sunday 05 September 2010] [06:09:26] <pieter_hintjens> should be nice and easy to implement
[Sunday 05 September 2010] [06:09:41] <sustrik> if you have resources to do that
[Sunday 05 September 2010] [06:09:58] <sustrik> an example?
[Sunday 05 September 2010] [06:10:00] <pieter_hintjens> well, it’s a question of what examples are the most valuable
[Sunday 05 September 2010] [06:10:24] <pieter_hintjens> would you rather have a name resolution service or a file transfer service over 0MQ?
[Sunday 05 September 2010] [06:10:27] <sustrik> ah, in the guide?
[Sunday 05 September 2010] [06:10:29] <pieter_hintjens> yeah
[Sunday 05 September 2010] [06:10:43] <sustrik> afaiu the examples in the guide are to be simple
[Sunday 05 September 2010] [06:10:53] <sustrik> file transfer is rather complex
[Sunday 05 September 2010] [06:10:56] <pieter_hintjens> sustrik: with 0MQ everything is simple :-)
[Sunday 05 September 2010] [06:11:03] <sustrik> i mean chopping the files
[Sunday 05 September 2010] [06:11:09] <sustrik> reassemblig them
[Sunday 05 September 2010] [06:11:14] <sustrik> etc.
[Sunday 05 September 2010] [06:11:38] <pieter_hintjens> maybe
[Sunday 05 September 2010] [06:12:00] <pieter_hintjens> actually it’s not that complex, really
[Sunday 05 September 2010] [06:12:06] <sustrik> it’s up to you
[Sunday 05 September 2010] [06:12:17] <pieter_hintjens> the sophisticated part is the stage / delivery semantics but that’s a neat demo of 0MQ IMO
[Sunday 05 September 2010] [06:12:24] <pieter_hintjens> we’ll see...
[Sunday 05 September 2010] [06:12:33] <sustrik> optinally, it can be a tutorial
[Sunday 05 September 2010] [06:12:40] <pieter_hintjens> yes
[Sunday 05 September 2010] [06:12:52] <pieter_hintjens> it reminded me that someone said “80% of messaging is still file transfer”
[Sunday 05 September 2010] [06:13:33] <pieter_hintjens> and it kind of seems that by covering the extreme other side of messaging (from low-latency) we prove the generality of the model
[Sunday 05 September 2010] [06:14:01] <pieter_hintjens> also it fits into the notion of distributed device management via shuffling config files around
[Sunday 05 September 2010] [06:14:05] * pieter_hintjens will think about it
[Sunday 05 September 2010] [06:14:11] <sustrik> ok
[Sunday 05 September 2010] [06:14:39] <pieter_hintjens> we will need a simple tutorial on how to make and submit git patches
[Sunday 05 September 2010] [06:14:57] <sustrik> enumerating the steps won’t do?
[Sunday 05 September 2010] [06:15:07] <pieter_hintjens> that would be a simple tutorial, then
[Sunday 05 September 2010] [06:15:10] <pieter_hintjens> it would do, yes
[Sunday 05 September 2010] [06:15:35] <sustrik> gitt diff, send as a main, put [PATCH] in the subject, state it’s under MIT
[Sunday 05 September 2010] [06:15:43] <sustrik> that’s it, no?
[Sunday 05 September 2010] [06:15:59] <sustrik> git, mail*
[Sunday 05 September 2010] [06:16:12] <pieter_hintjens> roughly, but some of the details cause problems
[Sunday 05 September 2010] [06:16:32] <pieter_hintjens> “Honestly? I spent too much time being stuck. I ended up on some crappy tutorial that said to type “git-format-patch”, when of course it was “git format-patch”. I got stuck on that until I trial-and-error’d.”
[Sunday 05 September 2010] [06:16:58] <sustrik> so far we don’t require formatted patches
[Sunday 05 September 2010] [06:17:05] <sustrik> so simple diff would do
[Sunday 05 September 2010] [06:17:11] <sustrik> not even git diff is needed
[Sunday 05 September 2010] [06:17:22] <sustrik> but it would be nice if people sent formatted patches
[Sunday 05 September 2010] [06:17:22] <pieter_hintjens> mato was saying something about signed-off-by
[Sunday 05 September 2010] [06:17:31] <sustrik> yup
[Sunday 05 September 2010] [06:17:33] <pieter_hintjens> indeed
[Sunday 05 September 2010] [06:17:56] <pieter_hintjens> for later...
[Sunday 05 September 2010] [06:18:14] <sustrik> would be nice to document it though
[Sunday 05 September 2010] [06:18:35] <pieter_hintjens> yes
[Sunday 05 September 2010] [06:18:46] <pieter_hintjens> did you update the contract page or shall I do that
[Sunday 05 September 2010] [06:18:53] <pieter_hintjens> i thought the way you expressed it in the email was very clear
[Sunday 05 September 2010] [06:19:07] <pieter_hintjens> but it needs to be written down, or it’ll get lost
[Sunday 05 September 2010] [06:20:16] <sustrik> ok, i’ll do that
[Sunday 05 September 2010] [06:20:44] <pieter_hintjens> what we found, in FFII work, was that it was usually better to edit the page first, then post to the list explaining the change
[Sunday 05 September 2010] [06:21:03] <pieter_hintjens> this is kind of a standard pattern to avoid losing stuff in the email list
[Sunday 05 September 2010] [06:25:33] <sustrik> ok
[Sunday 05 September 2010] [06:25:43] <sustrik> btw, the inproc example
[Sunday 05 September 2010] [06:26:04] <sustrik> you can fix it by launch step1’s thread from step2
[Sunday 05 September 2010] [06:26:13] <sustrik> that should synchronise it
[Sunday 05 September 2010] [06:27:39] <pieter_hintjens> hmm, nice solution...
[Sunday 05 September 2010] [06:28:50] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [06:31:39] <pieter_hintjens> ack, that works
[Sunday 05 September 2010] [06:31:45] <sustrik> great
[Sunday 05 September 2010] [06:35:44] <guido_g> hi
[Sunday 05 September 2010] [06:35:58] <guido_g> just saw the patch thing
[Sunday 05 September 2010] [06:36:25] <guido_g> I used git format-patch
[Sunday 05 September 2010] [06:36:44] <guido_g> but i had my changes in a local branch
[Sunday 05 September 2010] [06:37:52] <sustrik> guido_g: hi
[Sunday 05 September 2010] [06:37:59] <sustrik> i am not an expert on git
[Sunday 05 September 2010] [06:38:12] <sustrik> do you need a local branch to be able to format a patch?
[Sunday 05 September 2010] [06:38:47] <guido_g> i’m no expert neither, but will have a look
[Sunday 05 September 2010] [06:39:20] <sustrik> looks like there’s a “single-commit” mode
[Sunday 05 September 2010] [06:42:52] <guido_g> git format-patch -k –stdout R1..R2 <- from the examples of git help format-patch
[Sunday 05 September 2010] [06:43:08] <guido_g> seems that it works for commits and ranges of commits
[Sunday 05 September 2010] [06:43:32] <sustrik> nice
[Sunday 05 September 2010] [06:43:38] <guido_g> git format-patch -3 <- sweet
[Sunday 05 September 2010] [06:43:54] <guido_g> last three commits as patches
[Sunday 05 September 2010] [06:44:22] <sustrik> bingo
[Sunday 05 September 2010] [06:44:28] <sustrik> that’s what most people need
[Sunday 05 September 2010] [06:44:36] <guido_g> uh
[Sunday 05 September 2010] [06:45:06] * guido_g thinks changes should be done in branches
[Sunday 05 September 2010] [06:46:11] <sustrik> they should, but you don’t expect newbies submitting patches to deal with branching
[Sunday 05 September 2010] [06:46:29] <sustrik> installing git and learning how to commit is hard enough
[Sunday 05 September 2010] [06:46:54] <sustrik> that’s why i am happy with applying simple git diffs for now
[Sunday 05 September 2010] [06:47:20] <sustrik> or even dumb old diffs
[Sunday 05 September 2010] [06:47:55] <guido_g> ok
[Sunday 05 September 2010] [06:48:07] <sustrik> step-by-step would be nice though
[Sunday 05 September 2010] [06:48:08] <guido_g> so I spend hours for nothing...
[Sunday 05 September 2010] [06:48:10] <guido_g> :)
[Sunday 05 September 2010] [06:48:18] <sustrik> i mean, that’s the goal
[Sunday 05 September 2010] [06:48:25] <sustrik> we have to educate people
[Sunday 05 September 2010] [06:48:29] <guido_g> right
[Sunday 05 September 2010] [06:48:47] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [06:48:48] <sustrik> do we have a patch process described anywhere?
[Sunday 05 September 2010] [06:48:57] <sustrik> hm
[Sunday 05 September 2010] [06:49:05] * guido_g runs for cover
[Sunday 05 September 2010] [06:49:59] <pieter_hintjens> gudio_g: i think branches are orthogonal to patches
[Sunday 05 September 2010] [06:50:11] <pieter_hintjens> in the simplest case, you clone a repo, make changes, do a diff or git diff, and get a patch
[Sunday 05 September 2010] [06:50:34] <pieter_hintjens> well, that’s the medium hard case
[Sunday 05 September 2010] [06:50:53] <pieter_hintjens> in the simplest case you take the source code package, make changes, do a diff and get a non-git patch
[Sunday 05 September 2010] [06:51:13] <pieter_hintjens> but that only applies to projects that make source code packages
[Sunday 05 September 2010] [06:51:20] <sustrik> ack, and the hard case requires a topc branch
[Sunday 05 September 2010] [06:51:26] <sustrik> topic*
[Sunday 05 September 2010] [06:52:27] <pieter_hintjens> yes, exactly
[Sunday 05 September 2010] [06:53:07] <pieter_hintjens> for projects like zguide, there is no source package
[Sunday 05 September 2010] [06:53:24] <pieter_hintjens> we have to educate people, as you say
[Sunday 05 September 2010] [06:54:05] <sustrik> we are at the beginning of the road, it’ll take some time to make this work
[Sunday 05 September 2010] [06:54:13] <sustrik> but it’s worth of doing
[Sunday 05 September 2010] [06:55:38] <sustrik> guido_g: can you write down your experience with format-patch?
[Sunday 05 September 2010] [06:56:35] <sustrik> it could be used as a basis for the patch process tutorial
[Sunday 05 September 2010] [06:57:54] <guido_g> hmmm
[Sunday 05 September 2010] [06:58:11] <guido_g> i did it by book, three steps and done
[Sunday 05 September 2010] [06:58:22] <guido_g> but if you really want it...
[Sunday 05 September 2010] [06:58:37] <sustrik> yes, please
[Sunday 05 September 2010] [06:58:43] <sustrik> you’ll spare me an hour :)
[Sunday 05 September 2010] [06:58:46] <pieter_hintjens> guido_g: yes please, we’ll have to document the whole thing
[Sunday 05 September 2010] [06:58:50] <guido_g> sigh
[Sunday 05 September 2010] [06:58:52] <pieter_hintjens> how to submit patch from source package
[Sunday 05 September 2010] [06:59:09] <pieter_hintjens> how to submit patch from git repository clone
[Sunday 05 September 2010] [06:59:12] <guido_g> ok, then i’ll write down what I did
[Sunday 05 September 2010] [06:59:17] <sustrik> thanks
[Sunday 05 September 2010] [06:59:18] <pieter_hintjens> how to make topic branches and then submit them as patches
[Sunday 05 September 2010] [06:59:37] <pieter_hintjens> can you create a page on the wiki, minimal, and we can take it from there?
[Sunday 05 September 2010] [06:59:52] <pieter_hintjens> you should be able to create pages in the docs: category
[Sunday 05 September 2010] [07:00:06] <pieter_hintjens> “we” = everyone who cares about this
[Sunday 05 September 2010] [07:01:02] <guido_g> this cross-domain cookie thing sucks
[Sunday 05 September 2010] [07:02:38] <sustrik> ah, yes
[Sunday 05 September 2010] [07:02:53] <sustrik> pieter, any idea what’s the problem?
[Sunday 05 September 2010] [07:02:56] <guido_g> ok, where’s the how-to for the wiki?
[Sunday 05 September 2010] [07:03:09] <guido_g> sustrik: it’s a wikidot thing
[Sunday 05 September 2010] [07:03:12] <sustrik> guido_g: just post the command lines here
[Sunday 05 September 2010] [07:03:18] <guido_g> ok
[Sunday 05 September 2010] [07:11:33] <guido_g> that’s the whole magic
[Sunday 05 September 2010] [07:12:26] <guido_g> “git format-patch -1” is really nice
[Sunday 05 September 2010] [07:13:22] <sustrik> thanks!
[Sunday 05 September 2010] [07:16:40] <jond> hi
[Sunday 05 September 2010] [07:16:47] <sustrik> jon: hi
[Sunday 05 September 2010] [07:16:59] <jond> good to see a policy round the patches
[Sunday 05 September 2010] [07:17:11] <sustrik> ack
[Sunday 05 September 2010] [07:17:12] <sustrik> will you be able to make it for the meetup on tuesday?
[Sunday 05 September 2010] [07:17:35] <jond> I’ve had trouble squahsing commits in the past, which are meant to be easier for maintainer
[Sunday 05 September 2010] [07:17:44] <sustrik> i would like to have a talk with you about the subscription forwarding
[Sunday 05 September 2010] [07:18:07] <jond> tues: should be ok, though had to re-arrange a family thing
[Sunday 05 September 2010] [07:18:20] <sustrik> ah, sorry about that
[Sunday 05 September 2010] [07:19:16] <sustrik> discussing development via emails takes much more time than having a private talk
[Sunday 05 September 2010] [07:19:16] <jond> also; i can’t see irc in the dayjob, or twitter etc so if there’s any last minute change mailing list is best
[Sunday 05 September 2010] [07:19:41] <sustrik> jond: ok, will do
[Sunday 05 September 2010] [07:21:22] <jond> on the subject of git, the chacon book seems to recommend git pull requests and format patch. it’s all very powerful but seems easy to hang yrself
[Sunday 05 September 2010] [07:22:08] <sustrik> we’ll refine the process as we get better in using git
[Sunday 05 September 2010] [07:22:09] <guido_g> format-patch itself is quite nice, as shown above
[Sunday 05 September 2010] [07:22:35] <jond> am I meant to bring my local patch branches upto date with master/maint before running the diff/format-patch?
[Sunday 05 September 2010] [07:23:01] <sustrik> jond: that would be nice
[Sunday 05 September 2010] [07:23:31] <sustrik> otherwise it’s up to me to resolve the conflicts
[Sunday 05 September 2010] [07:23:40] <sustrik> which can in turn break your code
[Sunday 05 September 2010] [07:24:12] <sustrik> well, it depends
[Sunday 05 September 2010] [07:24:25] <sustrik> if you are submitting a single-line patch
[Sunday 05 September 2010] [07:24:34] <sustrik> and there have been a lot of change on master
[Sunday 05 September 2010] [07:24:46] <sustrik> it’s probably easier for me to merge it then for you
[Sunday 05 September 2010] [07:24:50] <jond> I also think we should recommend squashing the commits, as it makes the patch simpler to read (if git rebase -i works that is)
[Sunday 05 September 2010] [07:25:25] <sustrik> squash = rebase to the actual head?
[Sunday 05 September 2010] [07:25:55] <guido_g> already going way to far
[Sunday 05 September 2010] [07:26:09] <jond> i am trying to keep in line with 2.0.9 and maint at the moment. what mato seems to have put in place is similar to waht we do with clearcase in the day job.
[Sunday 05 September 2010] [07:26:41] <guido_g> pull -> [branch or not] -> changes -> commit [may be repeated] -> format-patch -> mailing list
[Sunday 05 September 2010] [07:27:59] <jond> well in the past, I used the rebase to make all the individual commits I did on my branch appear as 1 for delivery. It’s really annoying to have seperate commits for typo’s etc
[Sunday 05 September 2010] [07:28:41] <guido_g> but it’s great if you have different changes made in a row
[Sunday 05 September 2010] [07:28:52] <guido_g> so the maintainer can the ones he likes
[Sunday 05 September 2010] [07:29:25] <guido_g> anyhow, i think i’ll stick with this easy way
[Sunday 05 September 2010] [07:29:55] <jond> i’d use a seperate branch for each change.
[Sunday 05 September 2010] [07:30:33] <guido_g> if you do large changes, that might be ok
[Sunday 05 September 2010] [07:30:53] <guido_g> for smaller ones it’s sounds like a nightmare
[Sunday 05 September 2010] [07:32:32] <sustrik> jond: i wouldn’t bother with maint branch for the development work
[Sunday 05 September 2010] [07:32:54] <sustrik> it’s meant rather for small fixes
[Sunday 05 September 2010] [07:34:04] <sustrik> guido_g: yes, the smaller the patch the simpler the process should be
[Sunday 05 September 2010] [07:34:26] <sustrik> that’s why i’m saying i would accept even plain old diffs
[Sunday 05 September 2010] [07:34:45] <guido_g> great
[Sunday 05 September 2010] [07:35:02] <guido_g> lowers the entry barrier
[Sunday 05 September 2010] [07:35:11] <sustrik> exactly
[Sunday 05 September 2010] [07:35:27] <sustrik> however, if you ar woking on new functionality that’ll take 1/2 yesr to complete
[Sunday 05 September 2010] [07:35:39] <sustrik> you definitely want your own branch
[Sunday 05 September 2010] [07:35:44] <guido_g> and i’d demand payment :)
[Sunday 05 September 2010] [07:35:51] <sustrik> good point :)
[Sunday 05 September 2010] [07:37:28] <guido_g> so, now that i burned my eggs for brunch while writing git examples, i’m in the right mood for writing some more tests...
[Sunday 05 September 2010] [07:37:38] <jond> the many branches thing pretty much also depends on the turnaround of the patch getting to master etc
[Sunday 05 September 2010] [07:38:40] <jond> guido_g : i should send you the prefix_tree test ( was on the mailing list, i think we need tests now)
[Sunday 05 September 2010] [07:39:19] <jond> sustrik: hopefully get chance to discuss that pub filtering on tuesday
[Sunday 05 September 2010] [07:41:12] <sustrik> jond: sorry, i’ve seen it, but i was busy since :(
[Sunday 05 September 2010] [07:42:10] Quit xla has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [07:43:41] <jond> yes me too; 0MQ stuff very intermittent , as dayjob is not involving it all.
[Sunday 05 September 2010] [07:44:13] <guido_g> sigh at least, I’m not alone...
[Sunday 05 September 2010] [07:44:23] <jond> incidently a SUB socket can bind can’t it and then mutiple PUB’s connect
[Sunday 05 September 2010] [07:46:35] <sustrik> jond: yes
[Sunday 05 September 2010] [07:47:13] <sustrik> thus the subscription should be forwarded to _all_ connected publishers
[Sunday 05 September 2010] [07:47:51] <jond> ok, then maybe that’s one for the docs, someone asked that last night on here, they wanted fan-in to single server, I don’t think we cover that in the docs.
[Sunday 05 September 2010] [07:48:39] <sustrik> it kind of follows from the orthogonality of socket types and binds/connects
[Sunday 05 September 2010] [07:48:46] Quit keffo has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [07:48:53] <sustrik> but yes, it would be nice to mention it somewhere
[Sunday 05 September 2010] [07:49:02] <sustrik> maybe in the guide?
[Sunday 05 September 2010] [07:49:40] <jond> sustrik: sorry I meant the guide
[Sunday 05 September 2010] [07:50:09] <sustrik> pieter_hintjens: see above, suggestion for the guide
[Sunday 05 September 2010] [07:51:19] <sustrik> btw, the fact that binds/connects are independent of socket types is something that most people don’t get immediately
[Sunday 05 September 2010] [07:55:22] <jond> yes, i think that is the problem. there’s that big book enterprise messaging or whatever be nice to lift some scenarios from there and put in guide in 0MQ terms, which is pretty much what ph has been doing very well.
[Sunday 05 September 2010] [07:56:23] <sustrik> yes, it seems to work
[Sunday 05 September 2010] [07:57:08] <sustrik> btw, i’ve already seen some job ads requiring 0MQ knowledge so hopefully we’ll get into the phase where people are actually paid for working on 0MQ
[Sunday 05 September 2010] [07:57:11] <sustrik> we’ll see
[Sunday 05 September 2010] [07:58:55] <jond> that’s interesting news. I can see it being difficult to get 0MQ in some places with established code (understandly so in some cases, like reliable multicast)
[Sunday 05 September 2010] [08:01:48] <sustrik> well, i don’t expect nasdaq to replace their infrastrucutre by 0mq any time soon :)
[Sunday 05 September 2010] [08:02:11] <sustrik> but small companies are more flexible in adopting new technologies
[Sunday 05 September 2010] [08:05:11] <guido_g> hopefully
[Sunday 05 September 2010] [08:05:37] <guido_g> would be nice to have a mq related day job
[Sunday 05 September 2010] [08:09:08] <sustrik> my guess is that the early adopters are stock trading start-ups
[Sunday 05 September 2010] [08:09:46] <sustrik> that would mean that places like NY, London or Chicago are most interesting
[Sunday 05 September 2010] [08:09:55] <sustrik> i may be wrong though
[Sunday 05 September 2010] [08:10:35] <guido_g> hmmm... with mongrel2 there might be even ordinary web-work available
[Sunday 05 September 2010] [08:10:51] <sustrik> interesting point
[Sunday 05 September 2010] [08:11:11] <sustrik> i haven’t thought of that as i never done anything with web myself
[Sunday 05 September 2010] [08:11:29] <guido_g> i left it in early 2005 and never looked back :)
[Sunday 05 September 2010] [08:11:41] <sustrik> :)
[Sunday 05 September 2010] [08:11:44] <guido_g> never did fronted things anyway
[Sunday 05 September 2010] [08:12:13] <lestrrat> I doubt mongrel2 would really generate that much 0mq jobs... not at least from the web developer’s standpoint.
[Sunday 05 September 2010] [08:12:37] <lestrrat> zeromq is completely abstracted away in the web framework layer, so the web developers won’t even know ;)
[Sunday 05 September 2010] [08:12:57] <sustrik> right – unless you want to scale
[Sunday 05 September 2010] [08:13:06] <sustrik> what if you want N web servers
[Sunday 05 September 2010] [08:13:07] <lestrrat> of course, some of the more curious guys will do interesting web stuff with 0mq
[Sunday 05 September 2010] [08:13:14] <sustrik> and load-balanced services
[Sunday 05 September 2010] [08:13:22] <sustrik> persistence in the middle, etc.
[Sunday 05 September 2010] [08:13:40] <sustrik> that are google-like scenarios afaics
[Sunday 05 September 2010] [08:14:04] <guido_g> would also be nice to have it for mor ebay like things
[Sunday 05 September 2010] [08:14:08] <guido_g>
*more
[Sunday 05 September 2010] [08:14:14] <lestrrat> well, you’re talking about the smart guys, they’ll do anything. your average joe web devs won’t touch it with a 40ft pole
[Sunday 05 September 2010] [08:14:16] <lestrrat> ;)
[Sunday 05 September 2010] [08:14:36] <guido_g> lestrrat: you’re wrong
[Sunday 05 September 2010] [08:14:42] <guido_g> they do touch it
[Sunday 05 September 2010] [08:14:48] <guido_g> which is the problem...
[Sunday 05 September 2010] [08:15:05] <lestrrat> I knew where that was going ;)
[Sunday 05 September 2010] [08:15:17] <sustrik> :)
[Sunday 05 September 2010] [08:15:38] <guido_g> hehe
[Sunday 05 September 2010] [08:24:37] Quit keffo has left this server (Ping timeout: 260 seconds).
[Sunday 05 September 2010] [08:31:23] <guido_g> q: where do i find the declaration of int64_t?
[Sunday 05 September 2010] [08:33:24] <sustrik> guido_g: stdint.h
[Sunday 05 September 2010] [08:33:36] <guido_g> thx
[Sunday 05 September 2010] [09:06:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [09:15:09] <jond> sustrik: yes, for the large orgs change more likely thru acquisition/merger so small is best for more leading stuff
[Sunday 05 September 2010] [09:16:08] <jond> re : mongrel2 does anyone understand how the 0MQ integration works? It uses that c based co-routine library?
[Sunday 05 September 2010] [09:18:08] Quit xla has left this server (Read error: Operation timed out).
[Sunday 05 September 2010] [09:37:22] <sustrik> jond: i think so, but i haven’t checked
[Sunday 05 September 2010] [11:02:38] Quit
xla_ has left this server (Ping timeout: 245 seconds).
[Sunday 05 September 2010] [11:12:42] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [11:40:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [11:54:32] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [11:57:20] Quit xla has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [12:27:54] Part andrewvc has left this channel.
[Sunday 05 September 2010] [12:31:10] Quit AndrewBC has left this server (Ping timeout: 272 seconds).
[Sunday 05 September 2010] [12:37:26] Quit keffo has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [13:25:56] <cremes> Tasser: i’m not on irc very much during the weekends (Chicago, IL, US); if you have questions i recommend you send them to the mailing list
[Sunday 05 September 2010] [13:37:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [14:05:27] <pieter_hintjens> sustrik: re, was out with the family in the woods...
[Sunday 05 September 2010] [14:08:03] <pieter_hintjens> i will cover publishers connecting to subscriber when i find a sensible use case
[Sunday 05 September 2010] [14:08:28] <pieter_hintjens> “this is what you can do” is not useful but “this is how you solve problem X” is useful
[Sunday 05 September 2010] [14:09:10] <pieter_hintjens> fan-in to single server... maybe for collecting statistics
[Sunday 05 September 2010] [14:09:18] <pieter_hintjens> though why not use PUSH rather than PUB...?
[Sunday 05 September 2010] [14:30:30] <cremes> pieter_hintjens: fyi, there are some comments on that igvita article that probably need a response
[Sunday 05 September 2010] [14:30:42] <pieter_hintjens> cremes: am actually doing that right now
[Sunday 05 September 2010] [14:30:57] <cremes> i would respond if i knew anything about boost::asio or sc-whatever
[Sunday 05 September 2010] [14:31:00] <cremes> cool
[Sunday 05 September 2010] [14:31:00] <pieter_hintjens> though I doubt they “need” a response except “download it and try it”
[Sunday 05 September 2010] [14:31:07] <cremes> ha
[Sunday 05 September 2010] [14:31:21] <pieter_hintjens> i’m reminded of the virgin declaring why sex is so pointless and messy
[Sunday 05 September 2010] [14:31:50] <cremes> it’s just a good idea to make sure comments like that don’t go unanswered especially when google turns up that article in 6 months
[Sunday 05 September 2010] [14:32:14] <pieter_hintjens> ack
[Sunday 05 September 2010] [14:32:26] <cremes> l8r
[Sunday 05 September 2010] [14:32:28] <pieter_hintjens> if you have specific questions you’d like me to pontificate on, send me the URLs
[Sunday 05 September 2010] [14:32:40] <cremes> ack :)
[Sunday 05 September 2010] [14:35:31] <sigmonsays> does zeroMQ implement AMQP?
[Sunday 05 September 2010] [14:35:38] <DasIch> sigmonsays: no
[Sunday 05 September 2010] [14:37:29] <sigmonsays> [newb] is there a doc that describes what protocols it uses? I mean i’m sure the same concepts exist that AMQP implements..
[Sunday 05 September 2010] [14:41:54] <pieter_hintjens> sigmonsays: rfc.zeromq.org but it’s out of date
[Sunday 05 September 2010] [14:42:09] <pieter_hintjens> this is one thing i have wanted to make for a while, a correct spec of the protocol
[Sunday 05 September 2010] [14:42:23] <pieter_hintjens> but it is very different to AMQP
[Sunday 05 September 2010] [14:42:56] <pieter_hintjens> AMQP has stupid clients talking to a smart broker
[Sunday 05 September 2010] [14:43:01] <pieter_hintjens> 0MQ has smart clients talking to each other
[Sunday 05 September 2010] [14:43:11] <pieter_hintjens> it means the wire level protocol can be a lot simpler
[Sunday 05 September 2010] [14:50:00] <zedas> jond: i do. it uses the coroutines and zmq_poll
[Sunday 05 September 2010] [14:52:38] <sigmonsays> So, if persistence were required, would that be something you’d implement on the sending side? How does the “broker” play a role if clients talk directly to each other? I’m having a hard time finding a doc that demonstrates a traditional setup
[Sunday 05 September 2010] [14:55:05] <sigmonsays> Thanks, will do
[Sunday 05 September 2010] [14:55:32] <sustrik> in short, there are so called “devices” which are basically “mini-brokers”; it you need a full fledged broker, there’s a project that allows you to plug rabbitmq into 0MQ network
[Sunday 05 September 2010] [14:56:32] <sigmonsays> Yah, I was scanning the amq C API to see what it implements. Quite sexy API =P
[Sunday 05 September 2010] [15:02:55] <sigmonsays> Is SCTP a supported transport?
[Sunday 05 September 2010] [15:03:18] <sustrik> it would be nice if it was, but no, it isn’t yet
[Sunday 05 September 2010] [15:04:39] <sustrik> actually, it used to be implemented in old versions of 0MQ but people weren’t much interested :|
[Sunday 05 September 2010] [15:05:20] <sustrik> it seems there’s still some kind of fear of using anything other than TCP
[Sunday 05 September 2010] [15:05:37] <pieter_hintjens> I proposed to the IETF that they use SCTP as the basis for WebSockets
[Sunday 05 September 2010] [15:05:42] <pieter_hintjens> the response was interesting
[Sunday 05 September 2010] [15:05:50] * sustrik is curious
[Sunday 05 September 2010] [15:06:06] <pieter_hintjens> something along the lines of “it’s not present on most boxes and therefore will never be used by anyone at all”
[Sunday 05 September 2010] [15:06:27] <sustrik> hm, right, it not on win boxes
[Sunday 05 September 2010] [15:06:31] <pieter_hintjens> yes indeed
[Sunday 05 September 2010] [15:06:43] <pieter_hintjens> end of story, pretty much
[Sunday 05 September 2010] [15:06:52] <Zao> Nor on most FreeBSD boxen, as it conflicts with things like VIMAGE, if you’re insane enough to want that.
[Sunday 05 September 2010] [15:07:38] <pieter_hintjens> i can’t login for some weird reason
[Sunday 05 September 2010] [15:09:14] <sustrik> hm, win64 comment
[Sunday 05 September 2010] [15:09:24] <sustrik> does anyone have a win64 installation here?
[Sunday 05 September 2010] [15:09:38] <sustrik> it should be easy to fix
[Sunday 05 September 2010] [15:09:45] <sustrik> if it doesn’t work
[Sunday 05 September 2010] [15:10:25] <pieter_hintjens> sorry, my virtual box is 32-bit...
[Sunday 05 September 2010] [15:10:46] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [15:12:40] <jond> ph: the fan-in question was asked by twcc on irc about 20:00-21:00 last night. they wanted multiple publisher to single server.
[Sunday 05 September 2010] [15:13:10] <pieter_hintjens> jond: do you recall the use case?
[Sunday 05 September 2010] [15:13:58] <pieter_hintjens> “anyone have a drill for me?”
[Sunday 05 September 2010] [15:14:01] <pieter_hintjens> “why?”
[Sunday 05 September 2010] [15:14:10] <pieter_hintjens> “i have a headache that won’t go away”
[Sunday 05 September 2010] [15:14:15] <jond> ph: shall i mail the irc, I didnt answer as I wasnt around; steve-o was
[Sunday 05 September 2010] [15:14:29] <Zao> sustrik: Does he refer to PGM being hard/impossible to build?
[Sunday 05 September 2010] [15:14:34] <jond> ph: you considering trepanation then?
[Sunday 05 September 2010] [15:14:40] <pieter_hintjens> well, exactly
[Sunday 05 September 2010] [15:14:54] <pieter_hintjens> before answering people’s questions about “how to do X”, ask them “why”...
[Sunday 05 September 2010] [15:14:57] <sustrik> zao: who are you referring to?
[Sunday 05 September 2010] [15:15:00] <pieter_hintjens> i can see the irc log, will check
[Sunday 05 September 2010] [15:15:32] <Zao> sustrik: I thought that by “win64” comment you referred to the lad talking about 64-bit windows in the reddit thread?
[Sunday 05 September 2010] [15:16:02] <pieter_hintjens> jond: bah, simple PUSH->PULL pattern would do it IMO
[Sunday 05 September 2010] [15:16:22] <sustrik> ah, yes, maybe he was referring to pgm rather than 0mq as such
[Sunday 05 September 2010] [15:16:32] <sustrik> i don’t have win32 box, cannot check
[Sunday 05 September 2010] [15:16:35] <sustrik> win64*
[Sunday 05 September 2010] [15:16:36] <jond> ph: IIRC correctly he wanted intermittent/transient clients sending no-ack-needed messages to single server. it was pretty vague
[Sunday 05 September 2010] [15:16:58] <Zao> sustrik: I’ve got a 64-bit Windows box, but I don’t trust mq on it at all.
[Sunday 05 September 2010] [15:17:10] <sustrik> Zao: why so?
[Sunday 05 September 2010] [15:17:13] <sustrik> does it fail?
[Sunday 05 September 2010] [15:17:40] <pieter_hintjens> jond: i just read it twice, he wants a push/pull fanin afaics
[Sunday 05 September 2010] [15:17:50] <pieter_hintjens> it’s covered in chapter 1 of the guide even
[Sunday 05 September 2010] [15:18:00] <pieter_hintjens> should be a standard answer to people who ask for help
[Sunday 05 September 2010] [15:18:14] <pieter_hintjens> “if you’ve not read ch1 & ch2 of the guide, do so and then come back”
[Sunday 05 September 2010] [15:18:15] <Zao> sustrik: I set up a PUB and bind. I set up another thread with a SUB and connect. I subscribe to all topics. I then send lots of short messages which the SUB gets, until after a few dozen, it just blocks in zmq::poll.
[Sunday 05 September 2010] [15:18:27] <Zao> sustrik: I’ve got to boil it down to a test case before I report it on the issue tracker.
[Sunday 05 September 2010] [15:18:36] <Zao> As well as testing it with another compiler.
[Sunday 05 September 2010] [15:18:50] <sustrik> Zao: great, thanks
[Sunday 05 September 2010] [15:19:00] <Zao> This is 32-bit VC10, by the way.
[Sunday 05 September 2010] [15:19:30] <sustrik> ack
[Sunday 05 September 2010] [15:21:18] <jond> ph: yes, but you can also do a sub that binds and a pub that connects. There’s more than one way at present, which is good.
[Sunday 05 September 2010] [15:21:59] <sustrik> ph: centralised logging scanrio?
[Sunday 05 September 2010] [15:22:56] Quit andrewvc has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [15:25:54] <sigmonsays> Is the concept of headers up to the user to implement or does something like “routing-key” exist? ideally I wouldn’t want to subscribe based on the serialized data, but the headers accompanying it.
[Sunday 05 September 2010] [15:26:38] <sigmonsays> ie, ZMQ_SUBSCRIBE example from the user guide
[Sunday 05 September 2010] [15:27:28] <Zao> As far as I understand it, the topic is the prefix part of the data.
[Sunday 05 September 2010] [15:28:21] <sigmonsays> Well I wouldn’t want to unpack a serialized message to inspect it is where i’m getting
[Sunday 05 September 2010] [15:28:42] <sigmonsays> I suspect I would prefix the compress data with HTTP like headers, but i thought i’m missing something..
[Sunday 05 September 2010] [15:29:16] <sustrik> you can place your topic into one message part, data to another
[Sunday 05 September 2010] [15:29:34] <sustrik> the matching is always done on the first message part
[Sunday 05 September 2010] [15:29:44] <sigmonsays> that’s what I was missing. Thanks
[Sunday 05 September 2010] [15:31:23] <jond> sustrik: i think logging would be scenario for that use of pub/sub
[Sunday 05 September 2010] [15:32:06] <sustrik> jond: that’s what ph asked for, no?
[Sunday 05 September 2010] [15:33:58] <sigmonsays> Is there anything to facilitate message persistence? I am thinking of a WAN setup where aggregation points exist in each pop. Is that where a queue would fit in?
[Sunday 05 September 2010] [15:34:43] <DasIch> fyi logbook (a python library for logging) does now provide support for a zeromq handler/subscriber
[Sunday 05 September 2010] [15:35:08] <sustrik> you want messages just to be offloaded to disk, or do you want to survive crash/shutdown?
[Sunday 05 September 2010] [15:36:41] <sigmonsays> sustrik, good question, You mean HA replication versus disk persistence? I just want the message to be delivered and not lost/corrupted/etc.
[Sunday 05 September 2010] [15:37:01] <sustrik> i mean swapping vs. storage
[Sunday 05 September 2010] [15:37:56] <sigmonsays> Well swap dies when the box does. So that’s not ideal
[Sunday 05 September 2010] [15:38:40] <sustrik> then you have to put something that persists messages into the network
[Sunday 05 September 2010] [15:38:42] <jond> sustrik: i think so. you could probably do a really nice demo with tokyo tyrant which is the reverse index lookup thing of tokyo cabinet. especially if log messages are text
[Sunday 05 September 2010] [15:39:12] <sustrik> i’ve laready meantioned that there’s a project that allows you to use RabbitMQ broker as a node in 0mq network
[Sunday 05 September 2010] [15:39:14] <pieter_hintjens> jond: sorry, was writing a post on reddit
[Sunday 05 September 2010] [15:39:32] <pieter_hintjens> sustrik: centralized logging... yes, perhaps
[Sunday 05 September 2010] [15:39:48] <pieter_hintjens> why is that not a push/pull then?
[Sunday 05 September 2010] [15:40:12] <pieter_hintjens> i mean, pubsub is a natural pattern with subscribers that select data
[Sunday 05 September 2010] [15:40:19] <pieter_hintjens> centralized logging is a sink pattern
[Sunday 05 September 2010] [15:40:36] <pieter_hintjens> i don’t like using knives as screwdrivers, though it works
[Sunday 05 September 2010] [15:40:46] <sustrik> think of the case you would add another application listening to log feed
[Sunday 05 September 2010] [15:40:47] <sigmonsays> sustrik, I see, rabbitmq broker seems like overkill for persistence. I could see using tyrant/cabinet – Was wondering if any facilities were already provided. Still newb here.
[Sunday 05 September 2010] [15:40:55] <sustrik> would you want the logs to be load balanced?
[Sunday 05 September 2010] [15:41:04] <sustrik> i don’t think so
[Sunday 05 September 2010] [15:41:06] <pieter_hintjens> sustrik: then every single app needs to connect to TWO sinks
[Sunday 05 September 2010] [15:41:08] <pieter_hintjens> that’s insane
[Sunday 05 September 2010] [15:41:22] <pieter_hintjens> or you use a broker/forwarder
[Sunday 05 September 2010] [15:41:36] <sustrik> but not streamer
[Sunday 05 September 2010] [15:41:37] <pieter_hintjens> but then you don’t connect pubs to subs
[Sunday 05 September 2010] [15:41:42] <pieter_hintjens> no, not streamer
[Sunday 05 September 2010] [15:41:46] <pieter_hintjens> broker, in that case
[Sunday 05 September 2010] [15:41:47] <sustrik> => it’s pub/sub
[Sunday 05 September 2010] [15:42:03] <pieter_hintjens> why not push=>pull?
[Sunday 05 September 2010] [15:42:15] <sustrik> because push/pull load balances
[Sunday 05 September 2010] [15:42:19] <sustrik> rather than distributes
[Sunday 05 September 2010] [15:42:22] <pieter_hintjens> yeah, if you want multiple sinks
[Sunday 05 September 2010] [15:42:30] <sustrik> right
[Sunday 05 September 2010] [15:42:31] <pieter_hintjens> you’d do a push-pull to the broker and then pub-sub to all sinks
[Sunday 05 September 2010] [15:42:38] Quit gebi has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [15:42:44] <pieter_hintjens> still not a use case for pub->sub connections
[Sunday 05 September 2010] [15:42:49] <sustrik> you can do that, but why?
[Sunday 05 September 2010] [15:43:00] <pieter_hintjens> because it’s a sink
[Sunday 05 September 2010] [15:43:03] <pieter_hintjens> not a subscriber
[Sunday 05 September 2010] [15:43:16] <pieter_hintjens> ... message queuing semantics work better
[Sunday 05 September 2010] [15:43:26] <pieter_hintjens> i don’t want to lose log messages if broker isn’t there
[Sunday 05 September 2010] [15:43:33] <pieter_hintjens> i don’t want subscriptions
[Sunday 05 September 2010] [15:43:42] <pieter_hintjens> i want everything from all my upstream nodes
[Sunday 05 September 2010] [15:43:44] <pieter_hintjens> it’s a sink
[Sunday 05 September 2010] [15:43:52] <pieter_hintjens> pubsub is just not accurate...
[Sunday 05 September 2010] [15:44:07] <pieter_hintjens> we do IMO need a new socket type
[Sunday 05 September 2010] [15:44:12] <pieter_hintjens> which is the bus
[Sunday 05 September 2010] [15:44:19] <sustrik> shrug, it depends on you, if you believe that pipelining is the right pattern for message logs go for it
[Sunday 05 September 2010] [15:44:22] <pieter_hintjens> ... as we discussed before wrt error messages internally
[Sunday 05 September 2010] [15:44:38] <pieter_hintjens> it’s not the right pattern, it’s part of the pattern
[Sunday 05 September 2010] [15:44:43] <sustrik> pub/sub is a bus
[Sunday 05 September 2010] [15:44:48] <jond> ph+sustrik: what if we said ‘audit’ rather than logging. centralised audit of all messages?
[Sunday 05 September 2010] [15:45:11] <pieter_hintjens> audit, yes, nothing can be lost
[Sunday 05 September 2010] [15:45:29] <pieter_hintjens> pubsub is a stream with no guaranteed start
[Sunday 05 September 2010] [15:45:38] <sustrik> then yes, you want something reliable, like req/rep or pipeline
[Sunday 05 September 2010] [15:45:43] Part RunningMike has left this channel.
[Sunday 05 September 2010] [15:45:54] <sustrik> that’s what most logs are
[Sunday 05 September 2010] [15:45:56] <pieter_hintjens> yes
[Sunday 05 September 2010] [15:45:58] <sustrik> audit is different
[Sunday 05 September 2010] [15:46:04] <pieter_hintjens> logs are audits
[Sunday 05 September 2010] [15:46:12] <Zao> Oh joy. Even more fun behaviour sneaks up. zmq::poll(NULL, 0, t) asserts as winsock select craps itself on maxfd=~0.
[Sunday 05 September 2010] [15:46:14] <pieter_hintjens> losing a single message can be fatal
[Sunday 05 September 2010] [15:46:31] <pieter_hintjens> Zao: FD_SETSIZE?
[Sunday 05 September 2010] [15:47:01] Quit mato has left this server (Ping timeout: 260 seconds).
[Sunday 05 September 2010] [15:47:03] <pieter_hintjens> what is ~0...?
[Sunday 05 September 2010] [15:47:07] <sustrik> nope, zero-sized pollset
[Sunday 05 September 2010] [15:47:18] Quit keffo has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [15:47:23] Quit andrewvc has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [15:47:24] <sustrik> win32 select obviously don’t handle it well
[Sunday 05 September 2010] [15:47:36] * pieter_hintjens vaguely remembers something about that
[Sunday 05 September 2010] [15:47:38] <Zao> pieter_hintjens: maxfd is initialized to zmq::retired_fd and never gets assigned to anything else, as itemcount is 0.
[Sunday 05 September 2010] [15:47:58] <sustrik> Zao: can you report it please?
[Sunday 05 September 2010] [15:47:59] <Zao> winsock whines about invalid parameter, and the only one looking shady is the maxfd.
[Sunday 05 September 2010] [15:48:06] <Zao> sustrik: Yeah.
[Sunday 05 September 2010] [15:48:10] <sustrik> thanks
[Sunday 05 September 2010] [15:51:58] <pieter_hintjens> it looks to me that logging is a good usecase for a broker in fact
[Sunday 05 September 2010] [15:52:09] <pieter_hintjens> i’ll add this to the list of examples for the guide
[Sunday 05 September 2010] [15:55:39] <pieter_hintjens> sustrik: would it be fair to generalize that any N-to-N architecture needs a device in the middle?
[Sunday 05 September 2010] [15:55:49] <sustrik> definitely
[Sunday 05 September 2010] [15:55:53] <pieter_hintjens> except multicast, perhaps
[Sunday 05 September 2010] [15:55:55] <pieter_hintjens> ok
[Sunday 05 September 2010] [15:56:09] <sustrik> even with multicast you need a network switch in the middle
[Sunday 05 September 2010] [15:56:15] <pieter_hintjens> yes!
[Sunday 05 September 2010] [15:56:21] <sustrik> which is actaully a HW forwarder device
[Sunday 05 September 2010] [15:56:25] <pieter_hintjens> this is worth stating explicitly, it’s not clear initially why we need devices
[Sunday 05 September 2010] [15:56:32] <pieter_hintjens> i shall do this
[Sunday 05 September 2010] [15:56:37] <sustrik> ok
[Sunday 05 September 2010] [15:57:39] <Zao> Interesting. My testcase code doesn’t break.
[Sunday 05 September 2010] [15:57:42] <Zao> That’s annoying :D
[Sunday 05 September 2010] [15:57:58] <sustrik> you mean the poll bug?
[Sunday 05 September 2010] [15:58:14] <sustrik> it’s obvious what’s going on there, no need for test case
[Sunday 05 September 2010] [15:58:19] <sustrik> just report it
[Sunday 05 September 2010] [16:00:39] <Zao> sustrik: No, my other bug where poll on a SUB socket blocks even though the PUB peer keeps spamming it.
[Sunday 05 September 2010] [16:00:51] <Zao> sustrik: The one I’m trying to drum up a test case for, which doesn’t depend on D3D11.
[Sunday 05 September 2010] [16:01:00] <sustrik> ah
[Sunday 05 September 2010] [16:02:16] <pieter_hintjens> sustrik: you realize we’re totally screwed?
[Sunday 05 September 2010] [16:02:33] <sustrik> why so?
[Sunday 05 September 2010] [16:02:48] <pieter_hintjens> lol :-)
[Sunday 05 September 2010] [16:02:59] <pieter_hintjens> 10pm on a Sunday night and this is the most fun place to be
[Sunday 05 September 2010] [16:03:06] <sustrik> good god
[Sunday 05 September 2010] [16:03:09] <pieter_hintjens> yeah
[Sunday 05 September 2010] [16:03:20] <pieter_hintjens> and 0MQ is still under the radar...
[Sunday 05 September 2010] [16:03:47] <sustrik> i’ve been at a party, but returned to where fun is
[Sunday 05 September 2010] [16:03:58] <pieter_hintjens> splutter...!
[Sunday 05 September 2010] [16:04:02] <sustrik> :)
[Sunday 05 September 2010] [16:04:59] <pieter_hintjens> btw i added two concepts in the latest guide
[Sunday 05 September 2010] [16:05:17] <pieter_hintjens> 1. “0MQ string” -> length-delimited string with no trailing null
[Sunday 05 September 2010] [16:05:34] <pieter_hintjens> all the examples now correctly read/write strings in that format when they use strings
[Sunday 05 September 2010] [16:06:40] <sustrik> ok
[Sunday 05 September 2010] [16:08:38] <Zao> Hrm. Docs say that it should completely ignore the maxfd parameter.
[Sunday 05 September 2010] [16:11:54] Join snerd has joined this channel (
~motk@fedora/robk).
[Sunday 05 September 2010] [16:12:11] <sustrik> Zao: i think it does
[Sunday 05 September 2010] [16:12:25] <sustrik> Win32 fd_set is not a real fd_set
[Sunday 05 September 2010] [16:12:37] <sustrik> instead it’s a vector of SOCKETs
[Sunday 05 September 2010] [16:12:56] <sustrik> it’s vector of size 0 which causes trouble imo
[Sunday 05 September 2010] [16:16:27] Quit pieter_hintjens has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:16:27] Quit
snerd_ has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:16:27] Quit ivan has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:18:53] <sigmonsays> How does using (e)pgm allows multicast to be used reliably?
[Sunday 05 September 2010] [16:19:10] <sustrik> check RFC3208
[Sunday 05 September 2010] [16:19:18] <sustrik> there’s a resend mechanism
[Sunday 05 September 2010] [16:19:57] <sigmonsays> rgr
[Sunday 05 September 2010] [16:22:39] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [16:38:14] Part pieter_hintjens has left this channel.
[Sunday 05 September 2010] [18:11:18] Quit jsimmons has left this server (Client Quit).
[Sunday 05 September 2010] [18:48:26] Quit DasIch has left this server (Read error: Operation timed out).
[Sunday 05 September 2010] [18:49:41] Quit keffo has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [19:32:05] Quit andrewvc has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [19:38:27] Part andrewvc has left this channel.
[Sunday 05 September 2010] [19:48:22] Quit keffo has left this server (Ping timeout: 255 seconds).
[Sunday 05 September 2010] [20:36:56] Quit DasIch has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [22:18:03] <mikejs> there’s no INSTALL file in the 2.0.9 tarball, despite it being referenced in the README
[Sunday 05 September 2010] [23:14:49] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 05 September 2010] [23:18:35] Quit iFire has left this server (Remote host closed the connection).
[Sunday 05 September 2010] [23:52:10] Quit sustrik has left this server (Quit: Leaving).
[Monday 06 September 2010] [01:10:36] <mato> mikejs: good catch, my fault for not testing one setting properly, i’ll fix the tarballs
[Monday 06 September 2010] [01:14:31] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [01:59:20] Quit AndrewBC has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [02:04:43] Quit bobs has left this server (Client Quit).
[Monday 06 September 2010] [02:11:43] Part bobs has left this channel.
[Monday 06 September 2010] [02:15:30] Part bobs has left this channel.
[Monday 06 September 2010] [02:35:04] Part bobs has left this channel.
[Monday 06 September 2010] [03:18:19] Part bobs has left this channel.
[Monday 06 September 2010] [03:55:50] Quit mcxx has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [04:16:08] <BooTheHamster> hi
[Monday 06 September 2010] [04:17:20] <mato> hi BooTheHamster
[Monday 06 September 2010] [04:17:26] <BooTheHamster> can I connect ZMQ socket to zero port number, e.g. “
tcp://localhost:0”?
[Monday 06 September 2010] [04:17:55] <mato> 0 is not a legal port number, so no :-)
[Monday 06 September 2010] [04:18:34] <BooTheHamster> ZMQ don’t get a random port no?
[Monday 06 September 2010] [04:18:47] <mato> ah, you want dynamically assigned ports on socket?
[Monday 06 September 2010] [04:18:52] <BooTheHamster> yes
[Monday 06 September 2010] [04:18:54] <mato> that’s a different question :)
[Monday 06 September 2010] [04:19:08] <BooTheHamster> sorry for my english :(
[Monday 06 September 2010] [04:19:12] <mato> no problem
[Monday 06 September 2010] [04:19:33] <mato> for dynamically assigned ports, that’s not possible at the moment but you can emulate it
[Monday 06 September 2010] [04:19:50] <mato> on the bind side, you’ll want to do a loop and try to bind to a random port in a portrange
[Monday 06 September 2010] [04:20:49] <mato> lines 588 onwards, “bind_to_random_port()”
[Monday 06 September 2010] [04:21:02] <mato> you’ll then want to save the port number you bound to, and connect to that
[Monday 06 September 2010] [04:21:16] <mato> it’s a hack, I hope to resolve this properly in 2.1.x
[Monday 06 September 2010] [04:21:21] <mato> but that is still some way away
[Monday 06 September 2010] [04:21:55] <BooTheHamster> great thanx, mato :)
[Monday 06 September 2010] [04:22:12] <mato> BooTheHamster: you’re welcome
[Monday 06 September 2010] [04:23:22] Quit BooTheHamster has left this server (Quit: Page closed).
[Monday 06 September 2010] [04:31:06] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [04:32:56] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [05:15:35] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [05:18:33] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 06 September 2010] [05:20:08] Quit rbraley has left this server (Ping timeout: 245 seconds).
[Monday 06 September 2010] [05:23:41] <Tasser> no tcl binding for 2.0 yet?
[Monday 06 September 2010] [06:17:38] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [07:44:01] <BooTheHamster> Hi. Can I use telnet (or something else) to debugging my application with ZMQ_REP socket?
[Monday 06 September 2010] [07:44:28] <keffo> I suggest a debugger.
[Monday 06 September 2010] [07:47:51] <BooTheHamster> I mean that I develop a server application, but don’t have a client yet. Can I sent a messages to opened ZMQ socket?
[Monday 06 September 2010] [07:48:21] <guido_g> write a simple client first then
[Monday 06 September 2010] [07:50:43] <BooTheHamster> thanx
[Monday 06 September 2010] [07:51:40] <guido_g> there is even simpler stuff in the tests directory on master
[Monday 06 September 2010] [07:52:48] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Monday 06 September 2010] [07:52:56] <BooTheHamster> thanx, guido_g.
[Monday 06 September 2010] [08:14:03] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [09:04:47] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [09:28:07] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [09:42:24] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [09:44:03] Quit iFire has left this server (Changing host).
[Monday 06 September 2010] [10:08:03] Nick
DasIch_ is now known as DasIch.
[Monday 06 September 2010] [11:28:02] <pieterh> “This sounds like just a big marketing gimmic,”
[Monday 06 September 2010] [11:28:52] <guido_g> reddit? yes...
[Monday 06 September 2010] [11:28:59] <guido_g> ]:->
[Monday 06 September 2010] [11:31:14] Quit plq has left this server (Ping timeout: 264 seconds).
[Monday 06 September 2010] [11:32:01] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Monday 06 September 2010] [12:27:38] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Monday 06 September 2010] [12:30:26] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 06 September 2010] [13:06:45] Quit plq has left this server (Ping timeout: 258 seconds).
[Monday 06 September 2010] [13:19:01] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [13:19:41] Quit xla has left this server (Ping timeout: 260 seconds).
[Monday 06 September 2010] [13:31:40] <sbahra> pieterh, in all honesty, I first heard about ZeroMQ from the multithreading magic article (which states things like “To scale further, or developer must switch to 100% lock-free algorithms for data sharing. He is now into the realm of black magic.”), and it made it seem like ZeroMQ was a marketing gimmick. :-P
[Monday 06 September 2010] [13:32:13] <sbahra> Now, we’re seriously looking at using ZeroMQ for the product I’m involved with. :-P
[Monday 06 September 2010] [13:32:17] <pieterh> sbahra :-)
[Monday 06 September 2010] [13:32:57] <pieterh> can you share what that product is?
[Monday 06 September 2010] [13:33:26] <sbahra> If we use it, I’ll let you know.
[Monday 06 September 2010] [13:33:34] <sbahra> pieterh, it’s a distributed high performance MTA.
[Monday 06 September 2010] [13:33:45] <pieterh> nice!
[Monday 06 September 2010] [13:34:02] <pieterh> i’ve thought for a while that 0MQ was ideal for clustered products
[Monday 06 September 2010] [13:34:14] <sbahra> Based on my basic readings so far, it seems like it.
[Monday 06 September 2010] [13:34:21] <sbahra> We’re currently using spread.
[Monday 06 September 2010] [13:34:31] * pieterh does not know spread
[Monday 06 September 2010] [13:34:40] <sbahra> Much older than 0MQ, I think. :-)
[Monday 06 September 2010] [13:35:12] <pieterh> yeah, found that via google...
[Monday 06 September 2010] [13:35:30] <pieterh> interesting...
[Monday 06 September 2010] [13:36:16] <pieterh> biggest difference I see so far is that Spread uses blue while 0MQ uses red
[Monday 06 September 2010] [13:37:05] <sbahra> “Spread currently supports application messages upto around 100Kbytes. Currently the exact size is not exported as part of the API.”
[Monday 06 September 2010] [13:37:15] <sbahra> pieterh, it doesn’t seem like that’s the case for 0MQ. :-P
[Monday 06 September 2010] [13:37:28] <pieterh> oh... their latency is measured in msec...
[Monday 06 September 2010] [13:37:31] <sbahra> pieterh, spread is very resilient.
[Monday 06 September 2010] [13:38:16] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [13:38:51] <pieterh> hmm, it’s interesting...
[Monday 06 September 2010] [13:38:53] <sbahra> Anyways, we’ll see if we switch over to 0MQ sometime next year.
[Monday 06 September 2010] [13:39:03] <sbahra> I should do some hackery dackery doe. Bye.
[Monday 06 September 2010] [13:39:05] <pieterh> covers a lot of the ground people complain about missing in 0MQ
[Monday 06 September 2010] [13:39:10] <pieterh> cyat
[Monday 06 September 2010] [13:58:18] Join icy has joined this channel (
~icy@lighttpd/icy).
[Monday 06 September 2010] [14:06:33] Quit pieterh has left this server (Ping timeout: 258 seconds).
[Monday 06 September 2010] [14:09:09] Nick
AndrewBC_ is now known as AndrewBC.
[Monday 06 September 2010] [14:18:48] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:21:27] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:29:32] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:31:40] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 06 September 2010] [14:41:19] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:48:21] Quit pieterh has left this server (Ping timeout: 260 seconds).
[Monday 06 September 2010] [15:48:55] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [16:30:15] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 06 September 2010] [16:40:48] Quit andrewvc has left this server (Client Quit).
[Monday 06 September 2010] [16:41:49] Quit gebi has left this server (Ping timeout: 240 seconds).
*** on Mon Sep 6 16:59:51 2010
[Monday 06 September 2010] [16:59:51] Topic The channel topic is “Welcome!”.
[Monday 06 September 2010] [16:59:57] Mode Channel modes: no messages from outside, topic protection
[Monday 06 September 2010] [16:59:57] Created This channel was created on 2010-02-05 10:44.
[Monday 06 September 2010] [17:46:04] Part bobs has left this channel.
[Monday 06 September 2010] [17:47:41] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [19:03:43] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 06 September 2010] [20:36:59] Quit DasIch has left this server (Ping timeout: 272 seconds).
[Monday 06 September 2010] [21:39:53] Quit plq has left this server (Read error: Operation timed out).
[Monday 06 September 2010] [23:23:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [00:24:09] Quit keffo has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [01:27:02] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [01:48:19] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 07 September 2010] [01:54:18] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 07 September 2010] [01:54:29] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [01:57:45] <lestrrat> I’m having problems using fork() and making my parent process talk to the child processes via 0mq. I’m getting segfaults after the child exits and while the parent is in recv()
[Tuesday 07 September 2010] [01:57:50] <lestrrat> is this supposed to work?
[Tuesday 07 September 2010] [02:55:23] <pieterh> sustrik: you there?
[Tuesday 07 September 2010] [03:55:17] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Tuesday 07 September 2010] [04:18:48] Quit soren has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [04:32:49] Quit gebi has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [04:33:49] Quit xrfang has left this server (Remote host closed the connection).
[Tuesday 07 September 2010] [04:49:37] Quit plq has left this server (Read error: Operation timed out).
[Tuesday 07 September 2010] [05:07:21] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 07 September 2010] [05:20:45] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Tuesday 07 September 2010] [05:28:32] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 07 September 2010] [05:51:33] <pieterh> anyone here felt that XREP and XREQ could use better names?
[Tuesday 07 September 2010] [05:52:03] <lestrrat> I just gave a talk to my coworkers about zeromq, and yes, better_names++
[Tuesday 07 September 2010] [05:52:11] <pieterh> I was thinking of ROUTE and FORWARD
[Tuesday 07 September 2010] [05:53:24] * lestrrat doesn’t quite ring a bell
[Tuesday 07 September 2010] [05:53:58] <pieterh> XREP creates routing envelopes around incoming messages and uses these on output to route replies back to original clients
[Tuesday 07 September 2010] [05:54:09] Join plq has joined this channel (
~plq@78.189.30.218).
[Tuesday 07 September 2010] [05:54:12] <pieterh> XREQ just forwards messages in both directions without touching them
[Tuesday 07 September 2010] [05:55:08] <pieterh> XREP really looks like a router... it’s the only socket type that lets you address specific connections
[Tuesday 07 September 2010] [05:55:11] <keffo> I like ‘route’ very much, not forward so much
[Tuesday 07 September 2010] [05:55:36] <pieterh> yeah, forward wasn’t inspired
[Tuesday 07 September 2010] [05:55:51] <pieterh> it has to be a verb
[Tuesday 07 September 2010] [05:56:09] <pieterh> that says “move stuff in both directions but don’t mess with it”
[Tuesday 07 September 2010] [05:56:30] <pieterh> PORT
[Tuesday 07 September 2010] [05:56:45] <pieterh> XFER
[Tuesday 07 September 2010] [06:01:44] Quit andrewvc has left this server (Client Quit).
[Tuesday 07 September 2010] [06:11:56] <pieterh> lestrrat: does that ring a bell?
[Tuesday 07 September 2010] [06:13:11] <lestrrat> thinking
[Tuesday 07 September 2010] [06:15:38] <lestrrat> hmm. I grok ROUTE, but not the mudem part :) but I don’t have a great alternative plan either
[Tuesday 07 September 2010] [06:15:45] <lestrrat> naming is hard, eh.
[Tuesday 07 September 2010] [06:17:03] <pieterh> well, mudem is a play on modem, modulator/demodulator...
[Tuesday 07 September 2010] [06:17:09] Quit AndrewBC has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [06:17:14] <keffo> pieterh, I have a ‘route-codec’ in my code...
[Tuesday 07 September 2010] [06:17:18] <lestrrat> yeah, I know
[Tuesday 07 September 2010] [06:17:36] <pieterh> i don’t like invented words but if one has to invent them they should be expressive
[Tuesday 07 September 2010] [06:18:18] <keffo> whichi encodes/decodes routes for an xreq
[Tuesday 07 September 2010] [06:18:39] <pieterh> keffo: sounds right
[Tuesday 07 September 2010] [06:18:48] <pieterh> creates and uses envelopes, right?
[Tuesday 07 September 2010] [06:20:47] <keffo> it just handles them, contains a payload the ‘enduser’ is interested in, and also has a sendroute function
[Tuesday 07 September 2010] [06:20:48] <pieterh> well, an alternative to mudem: dispatch
[Tuesday 07 September 2010] [06:21:21] <keffo> naa, someone who implements any type of loadbalancing of messages is in essense a dispatcher, imo
[Tuesday 07 September 2010] [06:21:47] <pieterh> true
[Tuesday 07 September 2010] [06:22:06] <pieterh> it’s the combo of fanout and fanin
[Tuesday 07 September 2010] [06:23:06] <pieterh> i think “multiplex” is wrong since that suggest copying whereas its distribution
[Tuesday 07 September 2010] [06:23:40] <pieterh>
*it’s
[Tuesday 07 September 2010] [06:23:49] <lestrrat> yeah, I thought about multiplex, but it didn’t quite fit
[Tuesday 07 September 2010] [06:24:25] <pieterh> in terms of use cases, xreq is like push+pull, it ventilates and sinks at once
[Tuesday 07 September 2010] [06:25:24] <pieterh> one could create a nice pipeline pattern using just XREQ to XREQ
[Tuesday 07 September 2010] [06:27:11] <pieterh> how about... something more visual... 1TON
[Tuesday 07 September 2010] [06:27:57] <lestrrat> 1000kg!
[Tuesday 07 September 2010] [06:28:30] <pieterh> yeah
[Tuesday 07 September 2010] [06:28:41] <pieterh> 1-to-N for the pedantic of us
[Tuesday 07 September 2010] [06:31:05] <pieterh> it kind of feels more like a building block now
[Tuesday 07 September 2010] [06:42:46] <keffo> I have an issue where my worker process simply dissapears, but I cant seem to trap it
[Tuesday 07 September 2010] [06:42:57] <keffo> no exceptions, no atexits are run.. nada..
[Tuesday 07 September 2010] [06:42:58] <pieterh> what OS?
[Tuesday 07 September 2010] [06:42:59] <keffo> very annoying!
[Tuesday 07 September 2010] [06:43:01] <keffo> win7
[Tuesday 07 September 2010] [06:43:13] <pieterh> ah, that is a known problem
[Tuesday 07 September 2010] [06:43:21] <keffo> hu?
[Tuesday 07 September 2010] [06:43:23] <pieterh> the usual solution is to upgrade to Linux
[Tuesday 07 September 2010] [06:43:26] <pieterh> sorry :-)
[Tuesday 07 September 2010] [06:43:36] <keffo> caused by zmq??
[Tuesday 07 September 2010] [06:44:13] <keffo> tossing away 98% of the global userbase is hardly an upgrade btw :)
[Tuesday 07 September 2010] [06:44:14] <pieterh> I was kidding, my bad
[Tuesday 07 September 2010] [06:44:15] <pieterh> what language are you using?
[Tuesday 07 September 2010] [06:44:24] <keffo> c++, lua
[Tuesday 07 September 2010] [06:44:37] <pieterh> so you need a debug build of 0MQ IMO
[Tuesday 07 September 2010] [06:44:50] <keffo> oh it’s all debug, debugger is attached too :)
[Tuesday 07 September 2010] [06:44:58] <pieterh> aw :-(
[Tuesday 07 September 2010] [06:45:04] <keffo> gives me nothing.. I’ve tried all routes I can think of
[Tuesday 07 September 2010] [06:45:14] <keffo> abort()?
[Tuesday 07 September 2010] [06:45:20] <keffo> but why would that be called?
[Tuesday 07 September 2010] [06:45:20] <pieterh> DebugBreak() afair, then continue it in the debugger
[Tuesday 07 September 2010] [06:45:38] <pieterh> ... assertion failure?
[Tuesday 07 September 2010] [06:45:46] <keffo> nada :)
[Tuesday 07 September 2010] [06:46:12] <keffo> no asserts. no breakpoints, simply dissapears.. windows event log shows nothing
[Tuesday 07 September 2010] [06:46:16] <keffo> pussling actually
[Tuesday 07 September 2010] [06:46:21] <keffo> no exceptions are raised either
[Tuesday 07 September 2010] [06:46:40] <keffo> It’s as if the app cleanly exits, except it cant since it”s a while(true)
[Tuesday 07 September 2010] [06:46:57] <pieterh> it could exit in another thread I guess
[Tuesday 07 September 2010] [06:47:24] <pieterh> i’ve not worked on win32 for ages... maybe someone else here can be more helpful
[Tuesday 07 September 2010] [06:47:30] <keffo> does zmq ever call exit?
[Tuesday 07 September 2010] [06:47:39] <keffo> (or abort)
[Tuesday 07 September 2010] [06:47:41] * pieterh egreps the source code...
[Tuesday 07 September 2010] [06:47:48] Quit rbraley has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [06:48:04] <pieterh> nada
[Tuesday 07 September 2010] [06:48:09] <pieterh> asserts, yes
[Tuesday 07 September 2010] [06:48:43] <pieterh> 98%? keffo, 2010 is the Year of Linux
[Tuesday 07 September 2010] [06:48:51] <pieterh> it’s no more than 97.85% by now
[Tuesday 07 September 2010] [06:49:51] <keffo> if my mom could use any generic desktop linux without calling me, then I”d agree :)
[Tuesday 07 September 2010] [06:50:10] <pieterh> hah, my mum actually does use linux and has for years...
[Tuesday 07 September 2010] [06:51:02] <pieterh> but then again she’s currently asking me how to hide her IP address so she can troll Anonymous so perhaps she’s not typical...
[Tuesday 07 September 2010] [06:51:37] <pieterh> keffo: if you can make a reproducible case, and chop it down, maybe we can reproduce it on another platform
[Tuesday 07 September 2010] [06:52:18] <keffo> lord no, that would take ages :)
[Tuesday 07 September 2010] [06:52:34] <keffo> I just want to somehow detect -when- it happens, and go from there, but so far I’ve been unable to
[Tuesday 07 September 2010] [06:52:54] <pieterh> then, my friend, you might have to resort to...
[Tuesday 07 September 2010] [06:53:02] <pieterh> if really you have no other option...
[Tuesday 07 September 2010] [06:53:25] <keffo> print? =)
[Tuesday 07 September 2010] [06:53:28] <pieterh> yeah :-)
[Tuesday 07 September 2010] [06:53:38] <pieterh> don’t forget the fflush (stdout);
[Tuesday 07 September 2010] [06:54:52] <keffo> It’s remarkably reproducable though
[Tuesday 07 September 2010] [06:55:00] <pieterh> well, that’s always good
[Tuesday 07 September 2010] [06:55:03] <keffo> 4th time I calculate pi, it dissapears
[Tuesday 07 September 2010] [06:55:24] <pieterh> hopefully it remains stable as you add hundreds of prints
[Tuesday 07 September 2010] [06:55:40] <pieterh> you’re calculating pi?
[Tuesday 07 September 2010] [06:56:12] Quit mcxx has left this server (Quit: Lost terminal).
[Tuesday 07 September 2010] [06:57:24] <keffo> printing is not the problem, I generate ~250k of logs on each run :)
[Tuesday 07 September 2010] [06:57:42] <keffo> pi yeah, easy and verifiable thing to calc distributed :)
[Tuesday 07 September 2010] [06:58:04] <pieterh> is there an algo for distributed pi calculation somewhere?
[Tuesday 07 September 2010] [06:59:03] <keffo> sure
[Tuesday 07 September 2010] [06:59:08] <keffo> tons of different I guess
[Tuesday 07 September 2010] [06:59:10] <pieterh> i had an idea for a supermassive 0MQ project... lol
[Tuesday 07 September 2010] [06:59:39] <pieterh> not original but who cares...
[Tuesday 07 September 2010] [06:59:40] <keffo> as did I :)
[Tuesday 07 September 2010] [06:59:43] <keffo> for i=self.beginspan, self.endspan do
[Tuesday 07 September 2010] [06:59:43] <keffo> localpi = localpi + (1.0 / (i * 4.0 + 1.0) )
[Tuesday 07 September 2010] [06:59:43] <keffo> localpi = localpi - (1.0 / (i * 4.0 + 3.0) )
[Tuesday 07 September 2010] [07:00:15] <keffo> do that for each subspan of some arbitrary length, then sum them all up and the answer * 4 is pi :)
[Tuesday 07 September 2010] [07:00:30] <pieterh> not 42? weird...
[Tuesday 07 September 2010] [07:00:36] <keffo> hehe
[Tuesday 07 September 2010] [07:00:50] <pieterh> aight, so if we have a server somewhere that distributes workloads, and a simple 0MQ client that accepts them...
[Tuesday 07 September 2010] [07:01:52] <pieterh> has surely been done dozens of times
[Tuesday 07 September 2010] [07:02:08] <keffo> I’m doing a fairly more complicated scenario, but yeah
[Tuesday 07 September 2010] [07:05:18] <keffo> and I guess you can figure out why I want something a bit more flexibe than roundrobin :)
[Tuesday 07 September 2010] [07:06:00] <pieterh> right now I’m writing examples on how to use XREP to do routing
[Tuesday 07 September 2010] [07:06:25] <keffo> good, it was messy :)
[Tuesday 07 September 2010] [07:06:32] <pieterh> how so?
[Tuesday 07 September 2010] [07:06:40] <pieterh> you mean no documentation on the envelopes etc.?
[Tuesday 07 September 2010] [07:06:41] <keffo> just not very nicely explained
[Tuesday 07 September 2010] [07:06:47] <pieterh> right...
[Tuesday 07 September 2010] [07:11:26] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [07:13:09] <keffo> I would explain it as a stack+req...
[Tuesday 07 September 2010] [07:13:30] <keffo> push, push, push, payload, then pop,pop,pop, payload on the other side
[Tuesday 07 September 2010] [07:14:48] <keffo> btw, what happens in a queue device if a client never reconnects? will the msg linger indef.?
[Tuesday 07 September 2010] [07:21:02] <keffo> btw, what happens in a queue device if a client never reconnects? will the msg linger indef.?
[Tuesday 07 September 2010] [07:21:46] <pieterh> hmm, you mean a reply?
[Tuesday 07 September 2010] [07:21:53] <pieterh> with or without identity?
[Tuesday 07 September 2010] [07:22:19] <pieterh> this is what 0MQ/2.1 is fixing
[Tuesday 07 September 2010] [07:22:34] <pieterh> it will wait in some cases, discard in other cases
[Tuesday 07 September 2010] [07:22:41] <keffo> in general, the whole reconnect business
[Tuesday 07 September 2010] [07:22:53] <keffo> if it goes into the queue but never out, what happens to it?
[Tuesday 07 September 2010] [07:23:15] <pieterh> well, the queue is per socket, eventually
[Tuesday 07 September 2010] [07:23:52] * pieterh wishes the description of this had been documented beforehand
[Tuesday 07 September 2010] [07:24:13] <pieterh> there is not yet a proper explanation of how the 2.1 socket close semantics should work
[Tuesday 07 September 2010] [07:24:15] <pieterh> afaik
[Tuesday 07 September 2010] [07:24:29] <keffo> client(100msgs) -> queuedev -> service, then back again, except the client is gone forever..
[Tuesday 07 September 2010] [07:24:50] <pieterh> anonymous clients -> messages get thrown away
[Tuesday 07 September 2010] [07:25:06] <pieterh> client with identity -> messages persist as long as service is running
[Tuesday 07 September 2010] [07:25:25] <keffo> ok
[Tuesday 07 September 2010] [07:25:35] <pieterh> 0MQ does have the concept of a connection going away
[Tuesday 07 September 2010] [07:26:11] <pieterh> otherwise PUB sockets for example would end up with horrid resource leaks
[Tuesday 07 September 2010] [07:26:59] <keffo> I need to introduce some sort of session.. if I have (known)client A, does a bunch of test junk(like pi), but aborts prematurely, but then reconnects to start some other type of job, I dont want to receive a bunch of old pi results :)
[Tuesday 07 September 2010] [07:27:42] <keffo> and I would need to be able to tell all parties involved to dump everything related to an “old” session as well
[Tuesday 07 September 2010] [07:28:18] Quit mato has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [07:28:28] Quit plq has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [07:28:41] <pieterh> keffo: this starts to be industrial design work
[Tuesday 07 September 2010] [07:29:06] <keffo> pieterh, what do you mean?
[Tuesday 07 September 2010] [07:29:57] <pieterh> i mean, what you’re making is heavy duty...
[Tuesday 07 September 2010] [07:30:04] <keffo> oh very much :)
[Tuesday 07 September 2010] [07:30:25] <keffo> it has fried my brain on may occasions.. Tons of papers of diagrams spread all over the place :)
[Tuesday 07 September 2010] [07:30:32] <pieterh> if you have budget to throw at it, i can recommend an industrial 0MQ designer like Mato here
[Tuesday 07 September 2010] [07:31:07] <keffo> the bulk of the work is not the transport & topology though
[Tuesday 07 September 2010] [07:31:15] <keffo> although that needs to obviously be stable
[Tuesday 07 September 2010] [07:31:48] <pieterh> well, you need an infrastructure that understands ‘sessions’
[Tuesday 07 September 2010] [07:32:01] <keffo> Sure, but that”s already handled
[Tuesday 07 September 2010] [07:32:22] <pieterh> what do you still need then?
[Tuesday 07 September 2010] [07:32:32] <pieterh> apart from the thing not crashing...
[Tuesday 07 September 2010] [07:32:37] <keffo> hehe
[Tuesday 07 September 2010] [07:32:47] <keffo> lingering data trying to reconnect for one.
[Tuesday 07 September 2010] [07:33:20] <keffo> oh and more liberal means of implementing loadbalancing, but I’ve made that point already :)
[Tuesday 07 September 2010] [07:34:02] <pieterh> well, load balancing using XREP routing is pretty clear, and will be nicely explained in Ch3 of the Guide
[Tuesday 07 September 2010] [07:34:19] <keffo> will be? =)
[Tuesday 07 September 2010] [07:34:29] <pieterh> is in progress if I was not chatting here :-)
[Tuesday 07 September 2010] [07:34:39] <pieterh> anything to do with maintaining overall state is a different kettle of chicken, though
[Tuesday 07 September 2010] [07:34:45] <keffo> But I think I know what that will say by now :)
[Tuesday 07 September 2010] [07:35:05] <pieterh> hopefully, yeah
[Tuesday 07 September 2010] [07:35:41] <keffo> what I’m doing is something I’ve thought about for years though, so it should work :)
[Tuesday 07 September 2010] [07:35:55] <keffo> zmq solved a big gaping questionmark though :)
[Tuesday 07 September 2010] [07:36:43] <keffo> I might be getting a job soon though, so dev on this wll sadly be sidetracked to weekends and evenings only though :/
[Tuesday 07 September 2010] [07:37:32] <pieterh> is it open source?
[Tuesday 07 September 2010] [07:38:22] <keffo> it might be eventually!
[Tuesday 07 September 2010] [07:38:53] <keffo> would benefit the lua community I guess
[Tuesday 07 September 2010] [07:38:58] <pieterh> well... i’ve learned two relevant things here having done software for way too long
[Tuesday 07 September 2010] [07:39:04] <pieterh> a. if it’s not open source it will die
[Tuesday 07 September 2010] [07:39:17] <pieterh> b. if you don’t start as open source you can’t make it work afterwards
[Tuesday 07 September 2010] [07:40:02] <keffo> I dont agree, and I’ve never done anything other than software :)
[Tuesday 07 September 2010] [07:40:10] <pieterh> cause it’s not about building code but about building community...
[Tuesday 07 September 2010] [07:40:43] <pieterh> good luck, anyhow
[Tuesday 07 September 2010] [07:41:19] <keffo> I wasnt thinking of opensource as in leveraging resources, but simply aiding someone else, when I’m done with it :)
[Tuesday 07 September 2010] [07:41:49] <pieterh> nah, without people who helped make the code, it dies as soon as your evenings and weekends aren’t available any more
[Tuesday 07 September 2010] [07:42:34] <pieterh> it’s not about leveraging resources but about software that lives past the “free time” of its creator
[Tuesday 07 September 2010] [07:43:00] <pieterh> imho
[Tuesday 07 September 2010] [07:43:02] <keffo> Oh, it’s not free time at all, but I need to work for a bit to not starve :)
[Tuesday 07 September 2010] [07:43:33] <pieterh> starving is not pleasant, no
[Tuesday 07 September 2010] [07:44:20] <keffo> Not so much starving, but keeping girlfriend happier :)
[Tuesday 07 September 2010] [07:45:03] <keffo> when this is done, it will take me & the other dude about 2-3 weeks to produce the actual product we’ll eventually sell.. -That- part is already planned and so forth..
[Tuesday 07 September 2010] [07:45:17] <keffo> And ones that happens, there is no benefit to keep the code not opensource
[Tuesday 07 September 2010] [07:46:12] <keffo> err, once..
[Tuesday 07 September 2010] [07:46:53] <pieterh> :-) good spellng is hrad somtimes
[Tuesday 07 September 2010] [07:55:59] <keffo> um, this is odd
[Tuesday 07 September 2010] [07:56:10] <keffo> I wonder if lua might freak a little at some weird binary
[Tuesday 07 September 2010] [08:13:08] Nick
DasIch_ is now known as DasIch.
[Tuesday 07 September 2010] [08:50:26] Quit jsimmons has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [09:21:23] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Tuesday 07 September 2010] [09:51:25] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6d4ffd9 10/ (src/fq.cpp src/lb.cpp): Bug in fq_t and lb_t (when used via ZMQ_EVENTS option) fixed -
http://bit.ly/cvOPzL[Tuesday 07 September 2010] [10:12:19] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [10:43:46] Whois keffo is a user on channels: #zeromq
[Tuesday 07 September 2010] [10:43:46] Whois keffo is online via lindbohm.freenode.net (Stockholm, Sweden).
[Tuesday 07 September 2010] [10:43:46] Whois keffo has been idle for 2 hours, 47 minutes, and 36 seconds.
[Tuesday 07 September 2010] [10:43:46] Whois keffo has been online since 2010-09-07 02:18:32.
[Tuesday 07 September 2010] [10:43:46] Whois keffo is logged in as Keffo.
[Tuesday 07 September 2010] [10:43:46] Whois End of WHOIS list.
[Tuesday 07 September 2010] [11:14:02] <CIA-20> zeromq2: 03Martin Sustrik 07master * rf374431 10/ src/pipe.hpp : get rid of ‘has virtual functions but non-virtual destructor’ warnings in pipe.hpp -
http://bit.ly/9Relxm[Tuesday 07 September 2010] [11:21:12] <Tasser> cremes, it’s more about the ruby part you wrote
[Tuesday 07 September 2010] [11:21:36] <cremes> Tasser: i’m around if you have questions
[Tuesday 07 September 2010] [11:21:55] <Tasser> cremes, oh, just asking for the big picture
[Tuesday 07 September 2010] [11:22:03] <Tasser> aka what is where, how to stuff flows
[Tuesday 07 September 2010] [11:22:17] <cremes> sure...
[Tuesday 07 September 2010] [11:22:28] <Tasser> and probably write it down into your git
[Tuesday 07 September 2010] [11:22:35] <Tasser> HACKING or something like that :-)
[Tuesday 07 September 2010] [11:22:46] <cremes> whatever i write here, i’ll clean up and add to the README
[Tuesday 07 September 2010] [11:23:20] <cremes> ZM::Reactor is a thread that contains a single ZMQ context
[Tuesday 07 September 2010] [11:23:45] <cremes> from this context, you can create any kind of socket
[Tuesday 07 September 2010] [11:23:59] <cremes> (stop me if i’m not answering your question)
[Tuesday 07 September 2010] [11:24:21] <cremes> during socket creation, you pass a ruby object that will act as that socket’s handler
[Tuesday 07 September 2010] [11:24:32] <Tasser> callback?
[Tuesday 07 September 2010] [11:24:41] <cremes> the handler should provide on_attach, on_writable and on_readable methods
[Tuesday 07 September 2010] [11:25:32] <cremes> the on_attach method is called right away and lets you set things up (kind of like a constructor)
[Tuesday 07 September 2010] [11:25:47] <Tasser> so why not #new ?
[Tuesday 07 September 2010] [11:25:52] <cremes> the on_readable and on_writable methods are called when the socket is polled for those events and finds them to be true
[Tuesday 07 September 2010] [11:26:09] <cremes> explain what you mean by “not #new”?
[Tuesday 07 September 2010] [11:27:58] <Tasser> create a new instance per socket, so call #new and on that instance #on_writable, #on_readable
[Tuesday 07 September 2010] [11:29:26] <cremes> the handler instance is just a regular ruby class that implements the 3 methods i mentioned
[Tuesday 07 September 2010] [11:29:56] <cremes> it has a constructor (def initialize(
*args) nil; end) just like any other class
[Tuesday 07 September 2010] [11:30:16] <Tasser> less abstraction than EM
[Tuesday 07 September 2010] [11:30:21] <cremes> you could use one instance of a class to manage multiple sockets; look at the one-handed-ping-pong example
[Tuesday 07 September 2010] [11:30:33] <Tasser> yeah, having that one here atm
[Tuesday 07 September 2010] [11:30:36] <cremes> yeah, EM is kind of confusing with the EM::Connection stuff
[Tuesday 07 September 2010] [11:31:18] <Tasser> meh, gotta go :-(
[Tuesday 07 September 2010] [11:31:25] <cremes> sure
[Tuesday 07 September 2010] [11:31:50] <cremes> i’m usually on irc from 8am to 5pm central standard time (gmt -6, i think)
[Tuesday 07 September 2010] [11:32:08] <cremes> ping me if you have more questions or send them to the 0mq ml
[Tuesday 07 September 2010] [11:33:20] <bbigras> Is there a way to build zeromq with mingw?
[Tuesday 07 September 2010] [11:33:47] <cremes> bbigras: luislavena (rubyinstaller.org guy) has been playing with that
[Tuesday 07 September 2010] [11:34:14] <cremes> he opened an issue on github to fix a problem he encountered
[Tuesday 07 September 2010] [11:34:24] <cremes> so as far as i know he succeeded
[Tuesday 07 September 2010] [11:34:32] <bbigras> cremes: nice, thanks!
[Tuesday 07 September 2010] [11:35:16] <bbigras> cremes: Do you know if anyone had success using zeromq with Qt without having to disable Qt’s signal/slot macros?
[Tuesday 07 September 2010] [11:36:11] <cremes> i haven’t heard anything about that, so no
[Tuesday 07 September 2010] [11:36:23] <cremes> you might try asking on the 0mq ML
[Tuesday 07 September 2010] [11:38:07] <bbigras> cremes: thanks
[Tuesday 07 September 2010] [12:08:37] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Tuesday 07 September 2010] [12:11:23] Quit plq has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [12:20:59] Quit xla has left this server (Quit: leaving).
[Tuesday 07 September 2010] [12:36:25] Quit mato has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [12:43:48] Quit mato has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [12:54:09] Quit gebi has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [13:08:12] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 07 September 2010] [13:47:19] Quit keffo has left this server (Ping timeout: 252 seconds).
[Tuesday 07 September 2010] [13:50:51] Quit psino has left this server (Quit: psino).
[Tuesday 07 September 2010] [14:12:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [14:17:07] <ModusPwnens> hi
[Tuesday 07 September 2010] [14:17:50] <ModusPwnens> Is there anyone in here that has used google protobufs with zeromq? I’m wondering what kind of throughput is normal when using google protobufs
[Tuesday 07 September 2010] [14:19:03] Quit psino has left this server (Changing host).
[Tuesday 07 September 2010] [14:20:19] <cremes> ModusPwnens: i recommend you write a small benchmark that serializes/deserializes your data
[Tuesday 07 September 2010] [14:20:31] <cremes> and see what the upper limit is on your message rate
[Tuesday 07 September 2010] [14:20:48] <cremes> then 0mq will have that as its upper limit for throughput
[Tuesday 07 September 2010] [14:28:16] Quit dos000 has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [14:41:24] <ModusPwnens> yeah i did this cremes
[Tuesday 07 September 2010] [14:41:40] <ModusPwnens> I am just wondering whether or not my results are expected
[Tuesday 07 September 2010] [14:42:11] <ModusPwnens> Hmm, well actually, I benchmarked it with zeromq too
[Tuesday 07 September 2010] [14:42:30] <ModusPwnens> so it’s timing how long it takes to send messages as well
[Tuesday 07 September 2010] [14:44:04] <cremes> ModusPwnens: yeah, take 0mq out of the equation to get an upper bound
[Tuesday 07 September 2010] [14:44:17] <cremes> then you can test with 0mq to see what kind of overhead it is introducing
[Tuesday 07 September 2010] [14:57:44] <ModusPwnens> Hmm, another thing, I am having somewhat surprising results with the remote/local throughput tests
[Tuesday 07 September 2010] [14:58:27] <ModusPwnens> I am just using localhost, but I only get around 200 Mb/s throughput, which seems low to me.
[Tuesday 07 September 2010] [14:59:26] <cremes> ModusPwnens: try increasing the message body size from 50 bytes (from the example you posted last week) to something larger
[Tuesday 07 September 2010] [14:59:44] <cremes> also, note that the remote/local tests are doing a ping pong with REQ/REP sockets
[Tuesday 07 September 2010] [15:00:06] <cremes> you could see higher throughput on a PUB socket
[Tuesday 07 September 2010] [15:01:07] <ModusPwnens> Yeah, I have tried with larger message sizes. 5000 byte and 2500 count messages
[Tuesday 07 September 2010] [15:01:38] <ModusPwnens> is 200mb/s normal on localhost? I would have thought it would be much much faster
[Tuesday 07 September 2010] [15:01:43] <cremes> and what did you see?
[Tuesday 07 September 2010] [15:01:50] <cremes> with the varying message sizes...?
[Tuesday 07 September 2010] [15:02:20] <ModusPwnens> I see about 200mb/s or 5000 messages/s
[Tuesday 07 September 2010] [15:02:35] <cremes> so you see the same throughput regardless of message size?
[Tuesday 07 September 2010] [15:03:24] <ModusPwnens> not really, when the parameters are smaller I see different values
[Tuesday 07 September 2010] [15:03:41] <ModusPwnens> when i said 200 earlier i was using these paremeters
[Tuesday 07 September 2010] [15:03:56] <cremes> so what does 200 MB/s represent? the best that you see, the average or the worst?
[Tuesday 07 September 2010] [15:04:04] <ModusPwnens> about the average
[Tuesday 07 September 2010] [15:04:10] <ModusPwnens> i see 180 sometimes, sometimes 220
[Tuesday 07 September 2010] [15:04:33] <cremes> this is on windows, right?
[Tuesday 07 September 2010] [15:04:45] <ModusPwnens> Yeah. I was wondering if it would be faster on linux
[Tuesday 07 September 2010] [15:05:10] <cremes> sometimes; it’s hard to draw conclusions because this stuff is so dependent on OS and the hardware
[Tuesday 07 September 2010] [15:05:27] <ModusPwnens> Yeah. I notice that the official results on the zeromq website are ridiculously high
[Tuesday 07 September 2010] [15:05:38] <ModusPwnens> but I’m thinking that’s because they have a godly computer with 12 gigs of ram and 4 cores
[Tuesday 07 September 2010] [15:05:59] <ModusPwnens> i only have 1 core on this computer
[Tuesday 07 September 2010] [15:06:13] <ModusPwnens> as well as only 3 gigs of ram
[Tuesday 07 September 2010] [15:06:25] <icy> size of ram does not matter, speed does
[Tuesday 07 September 2010] [15:06:33] <ModusPwnens> I know that has an effect, but I’m not sure how large an effect that would be.
[Tuesday 07 September 2010] [15:06:36] <cremes> that computer is ancient if it has only 1 core; i don’t think intel has shipped a 1-core desktop cpu since around 2006
[Tuesday 07 September 2010] [15:07:09] <ModusPwnens> it’s actually relatively new
[Tuesday 07 September 2010] [15:07:13] <ModusPwnens> amd sempron m100
[Tuesday 07 September 2010] [15:07:22] <ModusPwnens> which i think has only 1 core
[Tuesday 07 September 2010] [15:08:47] <cremes> amd is behind the curve; sorry :)
[Tuesday 07 September 2010] [15:09:11] <ModusPwnens> Heh, apparently.
[Tuesday 07 September 2010] [15:09:18] <ModusPwnens> Anyways, is that sort of throughput expected?
[Tuesday 07 September 2010] [15:09:36] <cremes> again, it’s dependent upon OS and hardware
[Tuesday 07 September 2010] [15:09:49] <ModusPwnens> Hmm. So it’s at least not abnormal?
[Tuesday 07 September 2010] [15:10:10] <cremes> if you have nothing to compare it to...?
[Tuesday 07 September 2010] [15:10:43] <ModusPwnens> Ah, well there you go i guess.
[Tuesday 07 September 2010] [15:11:23] <icy> well, what cpu usage do you get while benchmarking?
[Tuesday 07 September 2010] [15:11:51] <icy> maybe it’s just really slow ram :)
[Tuesday 07 September 2010] [15:12:35] <ModusPwnens> i get 100% CPU usage
[Tuesday 07 September 2010] [15:13:24] <ModusPwnens> Does that mean the RAM is just slow?
[Tuesday 07 September 2010] [15:13:41] <Steve-o> what parameters are you using, I can compare with a single core Xeon right now
[Tuesday 07 September 2010] [15:13:46] <ModusPwnens> ok
[Tuesday 07 September 2010] [15:13:50] <ModusPwnens> for the cpu usage test I just ran
[Tuesday 07 September 2010] [15:13:57] <ModusPwnens> I used
[Tuesday 07 September 2010] [15:14:01] <ModusPwnens> 5000 byte messages
[Tuesday 07 September 2010] [15:14:08] <ModusPwnens> and 250,000 message count
[Tuesday 07 September 2010] [15:14:17] <ModusPwnens> all on localhost
[Tuesday 07 September 2010] [15:15:07] <Steve-o> I get 39,031 msgs/s and 1561 Mb/s
[Tuesday 07 September 2010] [15:15:10] <guido_g> message size: 1024 [B]
[Tuesday 07 September 2010] [15:15:10] <guido_g> message count: 100000
[Tuesday 07 September 2010] [15:15:10] <guido_g> mean throughput: 381327 [msg/s]
[Tuesday 07 September 2010] [15:15:10] <guido_g> mean throughput: 3123.831 [Mb/s]
[Tuesday 07 September 2010] [15:15:22] <guido_g> also a notebook
[Tuesday 07 September 2010] [15:16:25] <ModusPwnens> Hmm, that is definitely much higher than what I am getting.
[Tuesday 07 September 2010] [15:16:38] <ModusPwnens> how much ram do you have and what kind of processor?
[Tuesday 07 September 2010] [15:16:50] <ModusPwnens> guido that is
[Tuesday 07 September 2010] [15:16:53] <guido_g> Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz
[Tuesday 07 September 2010] [15:17:12] <guido_g> ram usage is not a problem at all
[Tuesday 07 September 2010] [15:17:35] <guido_g> it’s more a matter of cache and latency
[Tuesday 07 September 2010] [15:17:56] <ModusPwnens> Latency shouldn’t really be a problem on localhost though...right?
[Tuesday 07 September 2010] [15:18:16] <guido_g> ram latency
[Tuesday 07 September 2010] [15:18:20] <guido_g> sigh
[Tuesday 07 September 2010] [15:19:06] <icy> and bandwidth
[Tuesday 07 September 2010] [15:19:19] <ModusPwnens> Oh..sorry about that, i misunderstood..
[Tuesday 07 September 2010] [15:19:58] <guido_g> message size: 1024 [B]
[Tuesday 07 September 2010] [15:19:58] <guido_g> message count: 10000
[Tuesday 07 September 2010] [15:19:58] <guido_g> mean throughput: 98001 [msg/s]
[Tuesday 07 September 2010] [15:19:58] <guido_g> mean throughput: 802.824 [Mb/s]
[Tuesday 07 September 2010] [15:20:01] <guido_g> via lan
[Tuesday 07 September 2010] [15:20:21] <guido_g> so obviously you’re on a dog slow machine
[Tuesday 07 September 2010] [15:20:47] <icy> or windows :P
[Tuesday 07 September 2010] [15:21:15] <guido_g> hrhrhr
[Tuesday 07 September 2010] [15:22:47] <icy> nice webserver choice for zeromq.org
[Tuesday 07 September 2010] [15:23:15] <ModusPwnens> Hmm thanks. I will try to procure another computer to test this on.
[Tuesday 07 September 2010] [15:23:49] <guido_g> icy: you mean wikidot?
[Tuesday 07 September 2010] [15:24:13] <icy> I mean the lighttpd part :)
[Tuesday 07 September 2010] [15:31:34] <icy> hm local_thr does not seem to do anything on my osx box
[Tuesday 07 September 2010] [15:47:59] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [15:49:05] <cremes> icy: you need to run local_thr and remote_thr as a pair; one is the client and the other is the server
[Tuesday 07 September 2010] [15:49:16] <icy> ah right, thx
[Tuesday 07 September 2010] [15:53:30] <icy> ouf, this thing just sent me 1gb into swap
[Tuesday 07 September 2010] [15:54:15] <icy> sending 1000000 1kb messages
[Tuesday 07 September 2010] [15:54:35] <icy> I guess they get buffered in ram
[Tuesday 07 September 2010] [15:57:29] <cremes> icy: the receiver must not have pulled them off the queue fast enough
[Tuesday 07 September 2010] [15:57:43] <cremes> when i run those tests on my system, memory usage is constant (no queueing)
[Tuesday 07 September 2010] [15:58:08] <cremes> why don’t you pastie the arguments you passed to both programs so we can comment
[Tuesday 07 September 2010] [15:59:54] <icy> maybe I should start local_thr before remote_thr :)
[Tuesday 07 September 2010] [16:00:02] <cremes> try lowering the 1 million to 10 thousand and monitor the memory size of the programs
[Tuesday 07 September 2010] [16:00:22] <cremes> yeah, start order is important...
[Tuesday 07 September 2010] [16:01:08] <icy> doing that I get it to work even though it hits into swap briefly (understandable, the sender will always be faster)
[Tuesday 07 September 2010] [16:02:21] <cremes> icy: not true; this test is using REQ/REP sockets so it should only have 1 message in flight at any given time
[Tuesday 07 September 2010] [16:02:33] <cremes> one sender should not be able to get ahead of the other
[Tuesday 07 September 2010] [16:03:21] <cremes> (i was wrong in my statement from 2:57; no queueing should occur)
[Tuesday 07 September 2010] [16:03:22] <icy> I get 40mb ram usage with 100k messages
[Tuesday 07 September 2010] [16:03:45] <cremes> are you running the C programs or using the samples from another language binding?
[Tuesday 07 September 2010] [16:04:38] <icy> perf/ <- the ones in there which are C I think
[Tuesday 07 September 2010] [16:05:03] <cremes> did you modify the code at all?
[Tuesday 07 September 2010] [16:05:37] <icy> no, downloaded tarball, ./configure, make and ran the apps
[Tuesday 07 September 2010] [16:05:49] <cremes> huh... what OS?
[Tuesday 07 September 2010] [16:05:53] <icy> osx
[Tuesday 07 September 2010] [16:06:03] <cremes> 2.0.9?
[Tuesday 07 September 2010] [16:06:14] <cremes> 0mq, that is
[Tuesday 07 September 2010] [16:06:19] <icy> yea
[Tuesday 07 September 2010] [16:06:27] <Samy> I see a lot of emphasis on lock-free algorithms on the ZeroMQ website.
[Tuesday 07 September 2010] [16:06:28] <cremes> weird
[Tuesday 07 September 2010] [16:06:36] <cremes> it should not have unbounded memory growth
[Tuesday 07 September 2010] [16:06:42] <cremes> you should file a bug
[Tuesday 07 September 2010] [16:06:42] <Samy> What lock-free objects does ZeroMQ use? Where would the source-code be for them?
[Tuesday 07 September 2010] [16:07:02] <Samy> The atomics interface seemed too simple to support more complex data structures, though I wasn’t looking at the right thing.
[Tuesday 07 September 2010] [16:07:21] <cremes> Samy: check out the y_pipe stuff... i believe that is where the lock-free algorithms are used though sustrik would know better (he wrote it)
[Tuesday 07 September 2010] [16:07:43] <Samy> cremes, cool. Does sustrik IRC?
[Tuesday 07 September 2010] [16:08:05] <cremes> Samy: yes... he’s usually in channel but he isn’t here right now
[Tuesday 07 September 2010] [16:08:21] <Samy> Ok, thank you.
[Tuesday 07 September 2010] [16:08:34] <icy> the real mem does not show all the memory it uses as I’m already several hundred mb into swap at the time the screenshot was made
[Tuesday 07 September 2010] [16:08:55] <cremes> icy: that shouldn’t be; i would open a bug and describe the problem
[Tuesday 07 September 2010] [16:09:09] <cremes> make sure to include 0mq release, OS, OS release, etc
[Tuesday 07 September 2010] [16:09:19] <icy> k
[Tuesday 07 September 2010] [16:10:47] <cremes> icy: hold on a sec...
[Tuesday 07 September 2010] [16:11:07] <cremes> were you doing local_thr or local_lat as your perf test?
[Tuesday 07 September 2010] [16:14:00] <icy> thr
[Tuesday 07 September 2010] [16:16:59] <cremes> the local_thr/remote_thr examples don’t make any sense
[Tuesday 07 September 2010] [16:17:19] <cremes> the remote_thr program is using a REQ socket while the local_thr is using a SUB socket
[Tuesday 07 September 2010] [16:17:23] <cremes> the two are not compatible
[Tuesday 07 September 2010] [16:18:52] <icy> I’m totally knew to zeromq, just saw this benchmark app in the src dir and thought I’d give it a go :)
[Tuesday 07 September 2010] [16:18:55] <cremes> nm... i’m looking at the wrong stuff
[Tuesday 07 September 2010] [16:18:57] <cremes> argh...
[Tuesday 07 September 2010] [16:19:22] <cremes> okay, so remote_thr uses a PUB socket while local_thr uses a SUB
[Tuesday 07 September 2010] [16:19:24] <cremes> that is correct
[Tuesday 07 September 2010] [16:19:29] <cremes> you need to start local_thr first
[Tuesday 07 September 2010] [16:19:50] <cremes> remote_thr will slam your system by publishing as fast as possible, so there will be queueing
[Tuesday 07 September 2010] [16:20:05] <cremes> (i was thinking of the local_lat/remote_lat examples which uses different socket types)
[Tuesday 07 September 2010] [16:20:48] <icy> I did start local_thr first and even after that execited, remote_thr was allocating ram
[Tuesday 07 September 2010] [16:21:09] <icy> s/execited/exited/
[Tuesday 07 September 2010] [16:21:20] <cremes> yeah, i’m looking at it now...
[Tuesday 07 September 2010] [16:23:20] <cremes> if everything is working correctly, remote_thr should exit first
[Tuesday 07 September 2010] [16:37:31] Quit cremes has left this server (Quit: cremes).
[Tuesday 07 September 2010] [16:44:32] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Tuesday 07 September 2010] [16:45:13] Quit cremes has left this server (Client Quit).
[Tuesday 07 September 2010] [16:45:21] Part andrewvc has left this channel.
[Tuesday 07 September 2010] [16:46:33] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Tuesday 07 September 2010] [16:55:32] <dermoth> I’m wondering if there’s an easy way to monitor a queue size on a zeromq worker... It doesn’t seems like there is anything in the API
[Tuesday 07 September 2010] [16:55:42] <dermoth> on a zeromq broker I mean
[Tuesday 07 September 2010] [16:55:46] <dermoth> i.e. a device
[Tuesday 07 September 2010] [17:02:24] <ModusPwnens> whoa
[Tuesday 07 September 2010] [17:02:31] <ModusPwnens> im getting a strange error with the benchmarking tests now
[Tuesday 07 September 2010] [17:04:44] <dermoth> my concern is that the PUSH workers may send more messages than can be processed by the PULL workers, eventually filling up the queue. This would be possible if the PULL workers sync their state to disk to avoid data loss...
[Tuesday 07 September 2010] [17:12:07] <ModusPwnens> hey does zeroMQ allocate memory for messages all at once?
[Tuesday 07 September 2010] [17:12:25] Quit plq has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [17:13:35] <ModusPwnens> Or maybe it is because I am using the publish subscribe topology so it just continuously creates messages and the receiving end is not fast enough..
[Tuesday 07 September 2010] [17:33:14] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 07 September 2010] [18:23:07] <cremes> ModusPwnens: yes to the second thing you said; the publisher outpaces the subscriber
[Tuesday 07 September 2010] [18:23:34] <cremes> dermoth: no, there is no way to fetch the queue size; check the mailing list for the reasons why
[Tuesday 07 September 2010] [18:23:48] <cremes> that topic has been raised and answered a bunch of times (someone should add it to the FAQ)
[Tuesday 07 September 2010] [18:24:15] <cremes> dermoth: also, check out HWM (high water mark) settings for the PUSH sockets
[Tuesday 07 September 2010] [18:24:53] <cremes> by setting HWM, it will block when the queue hits that message level (or return EAGAIN if you try sending with ZMQ_NOBLOCK)
[Tuesday 07 September 2010] [18:26:05] <dermoth> yes, but I need to know berore i’ll ve blocking... I guess I could test the latency though
[Tuesday 07 September 2010] [18:26:34] <cremes> dermoth: what do you mean that you need to know before?
[Tuesday 07 September 2010] [18:26:44] <cremes> and what does latency have to do with it?
[Tuesday 07 September 2010] [18:27:21] <dermoth> well, if my queues start filling up at peak times I want to react before all pushers block... my primary goal in using zmq is to avoid blocking
[Tuesday 07 September 2010] [18:28:04] <cremes> ok, then use send with ZMQ_NOBLOCK and test for EAGAIN; when you get it then you know you have hit your high water mark
[Tuesday 07 September 2010] [18:28:14] <cremes> and you can take whatever action is necessary
[Tuesday 07 September 2010] [18:29:03] <dermoth> cremes, if the queue fill up on the device, then there will be some latency between the time I push to the queue and the time my worked gets the event. I can push a special message and have whichever worker gets it respond to me, then I know the latency. it it rises thern I need more workers downstream
[Tuesday 07 September 2010] [18:29:53] <cremes> ok
[Tuesday 07 September 2010] [18:30:25] <cremes> you could also have another pair of sockets where each worker tells the server/pusher that it has received a message
[Tuesday 07 September 2010] [18:30:54] <cremes> using this “out of band” communication, you could publish only those messages that can be immediately handled by a worker
[Tuesday 07 September 2010] [18:31:25] <cremes> you would have at most 1 message in someone’s queue because you would not push another one until each one had been acknowledged
[Tuesday 07 September 2010] [18:31:48] <cremes> that seems better than trying to rely on some weird latency calculation that might not be trustworthy
[Tuesday 07 September 2010] [18:31:51] <dermoth> i’ll implement XREQ/XREP for thing I need to make sure a worker is getting it... the rest is high-throghtput stuff that can suffer a small percentage loss...
[Tuesday 07 September 2010] [18:32:28] <cremes> definitely take a look at HWM
[Tuesday 07 September 2010] [18:32:36] <cremes> i think it does what you need
[Tuesday 07 September 2010] [18:33:27] <dermoth> the point is that I don’t want to block on the sending side... HWM will be useful in logging error conditions, but I should never end up hitting this limit...
[Tuesday 07 September 2010] [18:34:21] <cremes> dermoth: sorry, but your requirements don’t make sense to me
[Tuesday 07 September 2010] [18:34:59] <cremes> if you could get the queue length, you would probably prevent your pusher from sending more messages if it hit some threshold, right?
[Tuesday 07 September 2010] [18:35:08] <cremes> if so, then this is exactly what you can do with HWM
[Tuesday 07 September 2010] [18:35:27] <cremes> couple HWM with NO_BLOCK and you’ll get your “signal” that there aren’t enough workers
[Tuesday 07 September 2010] [18:35:38] <dermoth> well I don’t really control the pusher... it logs as data comes it. I need to have enough pullers to absorb the data as it comes in
[Tuesday 07 September 2010] [18:35:38] <cremes> assuming you ever hit the HWM
[Tuesday 07 September 2010] [18:36:28] <cremes> so you don’t have any control over throttling the pusher?
[Tuesday 07 September 2010] [18:37:24] <dermoth> because if the pushers block, it’s not goung to work anymore and I will loose data - the point is that the pushhers have to be non-blocking. But i’ll figure out something... maybe even measuring the rss size of the process might work.
[Tuesday 07 September 2010] [18:37:54] <cremes> dermoth: let me say it again then.... use send with NO_BLOCK and test for EAGAIN
[Tuesday 07 September 2010] [18:37:58] <cremes> this will NOT BLOCK
[Tuesday 07 September 2010] [18:38:05] <dermoth> yes I get that ;)
[Tuesday 07 September 2010] [18:38:20] <cremes> so what’s the problem then? ;)
[Tuesday 07 September 2010] [18:39:53] <dermoth> what I mean, EAGIN is bad too - I will log it & alert on it, but I want to know before that happens... If I start getting latency cpikes when I turn worn downstream workers, or during pikes, even before filling up the queues all the way up to the pushers, I want to know it. But that’s fine, I’ll work with what I have ;)
[Tuesday 07 September 2010] [18:40:16] <cremes> ok, i see
[Tuesday 07 September 2010] [18:40:49] <cremes> let us know what you come up with or add your solution to the wiki
[Tuesday 07 September 2010] [18:41:13] <dermoth> Sure. btw, it having multiple parallel devices wiorking well? i.e. if one crash will it nicely fall back to the other one? Will my XREQ/XREP suffer any latency? More that 2-3 seconds avg per requests may become problematic
[Tuesday 07 September 2010] [18:42:04] <dermoth> here actually on the ends it more like a REQ/REP socket, byt the device will be XREQ/XREP to pass & load-balance the messages
[Tuesday 07 September 2010] [18:43:32] <cremes> a device is just a fancy packaging for 2 sockets to aid with load balancing
[Tuesday 07 September 2010] [18:43:46] <cremes> if you are worried about a device crashing, then messages in flight could be lost
[Tuesday 07 September 2010] [18:43:49] <dermoth> i’ll probably be running some tests anyway... Thanks
[Tuesday 07 September 2010] [18:44:23] <cremes> if you can’t handle data loss, then you need to add some code around all of this to ack/nak messages and retry when things timeout or disappear
[Tuesday 07 September 2010] [18:44:34] <cremes> none of that is built in to 0mq; you have to build it on top
[Tuesday 07 September 2010] [18:44:57] <dermoth> yes... a srever crashing is pretty rare, what i’m concerned about it if the system will keep on working. the REQ/REP are for where I can’t handle loss and I will loose the request if the device crash with my message
[Tuesday 07 September 2010] [18:45:07] <cremes> and unless you have a really slow link or are sending very large messages, i can’t imagine it would ever take 2-3 seconds to send a message
[Tuesday 07 September 2010] [18:45:38] <cremes> if you use XREQ/XREP and do explicit acks, then you could have things continue to work
[Tuesday 07 September 2010] [18:46:12] <cremes> if you use REQ/REP sockets, they enforce a very strict send/recv/send/recv pattern, so a crashed peer would not be recoverable
[Tuesday 07 September 2010] [18:46:21] <dermoth> well, i’m thinking possible tcp timeouts, etc. I would assube ZeroMQ is able to handle well peers, but I will test it anyway. Thanks!
[Tuesday 07 September 2010] [18:46:31] <cremes> ok, good luck
[Tuesday 07 September 2010] [18:46:50] <dermoth> to handle well down peers
[Tuesday 07 September 2010] [18:47:44] <cremes> dermoth: TCP timeouts are not exposed to you via the 0mq api
[Tuesday 07 September 2010] [18:48:15] <cremes> 0mq may see a connection disappear at the tcp level, but that doesn’t mean that the socket is no good
[Tuesday 07 September 2010] [18:48:31] <cremes> because a 0mq socket can be bound or connected to multiple endpoints
[Tuesday 07 September 2010] [18:48:50] <cremes> one endpoint failure does not cause the whole 0mq socket to fail
[Tuesday 07 September 2010] [18:49:28] <cremes> the 0mq socket will continue to load balance messages to any “surviving” endpoints
[Tuesday 07 September 2010] [18:51:34] <dermoth> ok, sounds good. FWIW, what I have in mind to mointor PUSH/PULL latency is to send a special message with an IP/port to reply withm and have the workers respond with an udp packet. I can release the check as a Nagios plugin like I usually do for my monitoring scripts, but obviously the other end is up to the developer to get right
[Tuesday 07 September 2010] [18:53:32] <cremes> sounds neat
[Tuesday 07 September 2010] [19:06:05] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Tuesday 07 September 2010] [19:07:38] Quit keffo has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [19:09:50] Quit GeekGod has left this server (Quit: GeekGod).
[Tuesday 07 September 2010] [19:22:50] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [19:31:47] Quit psino has left this server (Quit: psino).
[Tuesday 07 September 2010] [20:37:05] Quit DasIch has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [21:11:23] Quit ModusPwnens has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [21:46:30] Quit gebi has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [22:25:30] Quit
halfie_ has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [23:47:58] Quit miguelito has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [02:41:08] Quit ivan has left this server (Ping timeout: 258 seconds).
[Wednesday 08 September 2010] [03:07:00] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [03:07:12] Quit
halfie_ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [04:12:25] <CIA-20> zeromq2: 03Martin Sustrik 07master * r91ea204 10/ (13 files in 2 dirs): EINTR returned from the blocking functions -
http://bit.ly/aSdLhb[Wednesday 08 September 2010] [04:29:31] <lestrrat> sustrik: almost.... s/Daisuke Make/Daisuke Maki/ ;) But yes, thanks for the EINTR fix!
[Wednesday 08 September 2010] [04:39:46] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 08 September 2010] [04:47:44] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [04:53:08] Quit jsimmons has left this server (Quit: Leaving).
[Wednesday 08 September 2010] [05:00:48] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 08 September 2010] [05:25:30] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 08 September 2010] [06:33:47] Quit gebi has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [07:15:44] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 08 September 2010] [07:44:46] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [07:58:48] <keffo> :)
[Wednesday 08 September 2010] [08:05:09] Quit __moore__ has left this server (Read error: Operation timed out).
[Wednesday 08 September 2010] [08:07:28] <pieterh> keffo: if I can figure out wtf I can do with an XREP talking to an XREP, I’ll finish this part of the Guide today
[Wednesday 08 September 2010] [08:07:44] <pieterh> turns out least-recently used is indecently easy to do
[Wednesday 08 September 2010] [08:11:00] <keffo> That would ameliorate at least part of the problem :)
[Wednesday 08 September 2010] [08:11:40] <keffo> Except not ‘successful send’ in my case, but when the answer is returned :)
[Wednesday 08 September 2010] [08:12:36] <keffo> It wouldn’t be very difficult for me to specify routing manually though, bypassing the zmq lb altogether
[Wednesday 08 September 2010] [08:12:46] <keffo> but I need to finish heartbeat system first
[Wednesday 08 September 2010] [08:13:18] <pieterh> well, here’s my take on custom LB...
[Wednesday 08 September 2010] [08:13:18] <keffo> I’m thinking pub -> sub->xreq back to source, and that would give me both route & availability?
[Wednesday 08 September 2010] [08:13:33] <pieterh> you want to use a XREP socket to do the routing
[Wednesday 08 September 2010] [08:13:41] <pieterh> you can use various algorithms to drive it
[Wednesday 08 September 2010] [08:13:52] <pieterh> depending on the socket type it talks to
[Wednesday 08 September 2010] [08:14:26] <keffo> I’m thinking a priorityqueue of all known/active nodes, then ‘nudging’ the weights based on speed/performance
[Wednesday 08 September 2010] [08:14:54] <pieterh> e.g. if your workers use req sockets, you get a nice LRU
[Wednesday 08 September 2010] [08:15:14] <pieterh> you don’t need a priority queue, just the message from the worker saying “ready”
[Wednesday 08 September 2010] [08:15:31] <keffo> hum, explain how req gives lru?
[Wednesday 08 September 2010] [08:15:43] <pieterh> well, think of HTTP long poll
[Wednesday 08 September 2010] [08:15:59] <pieterh> flow is worker says “ready”, router provides it workload
[Wednesday 08 September 2010] [08:16:09] <pieterh> workers say “ready” when they’re done...
[Wednesday 08 September 2010] [08:16:21] <pieterh> ready messages come into queue, router takes one off each time
[Wednesday 08 September 2010] [08:20:39] <keffo> uh, must draw on paper..
[Wednesday 08 September 2010] [08:21:03] <pieterh> hey, let me post this section of Ch 3, see if it makes any sense at all
[Wednesday 08 September 2010] [08:21:06] <pieterh> :-)
[Wednesday 08 September 2010] [08:21:19] <keffo> I think I need two prioqueues to be honest, one for picking the best of available set of nodes, and one for handing the most urgent tasks
[Wednesday 08 September 2010] [08:21:51] <keffo> let me know when you finish it!
[Wednesday 08 September 2010] [08:22:47] <guido_g> playing the “naming awareness” card: this isn’t REQuest/REPly anymore then, right?
[Wednesday 08 September 2010] [08:24:31] <pieterh> nope, this ain’t kansas anymore
[Wednesday 08 September 2010] [08:24:44] <pieterh> this is “off-road”...
[Wednesday 08 September 2010] [08:27:12] <pieterh> basically it’s about using identities and constructing envelopes so that XREP will route as we want to other nodes
[Wednesday 08 September 2010] [08:27:53] <pieterh> I just posted what I have...
[Wednesday 08 September 2010] [08:31:23] Quit rossij has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [08:33:13] <keffo> looks good
[Wednesday 08 September 2010] [08:35:28] <keffo> except the pub/sub ctrl+c sample code needs linebreaks
[Wednesday 08 September 2010] [08:37:57] <keffo> hum, it gets progressively difficult to read after that :)
[Wednesday 08 September 2010] [08:37:58] <pieterh> what sample is that?
[Wednesday 08 September 2010] [08:38:03] <pieterh> :-)
[Wednesday 08 September 2010] [08:38:35] <keffo> something got weird halfway down the article, it’s still embedded in the code preview
[Wednesday 08 September 2010] [08:38:45] <pieterh> hang on, refresh the page
[Wednesday 08 September 2010] [08:38:55] <pieterh> there was a fault in the text, i already fixed it
[Wednesday 08 September 2010] [08:39:16] <keffo> ignore, reloaded :)
[Wednesday 08 September 2010] [08:39:28] <keffo> lots of good food for thought there
[Wednesday 08 September 2010] [08:39:37] <guido_g> ack
[Wednesday 08 September 2010] [08:39:44] <guido_g> fig 22, the dashes are misplaced
[Wednesday 08 September 2010] [08:40:09] * keffo bbll
[Wednesday 08 September 2010] [08:40:10] <pieterh> guido_g: this is ditaa being too clever
[Wednesday 08 September 2010] [08:40:21] <pieterh> it thinks ‘-‘ in text is me trying to draw a line
[Wednesday 08 September 2010] [08:40:33] <guido_g> ouch
[Wednesday 08 September 2010] [08:40:49] <pieterh> ill try a unicode n-dash or something
[Wednesday 08 September 2010] [08:40:59] <guido_g> so no dash between single and hop and a slash for req/rep :)
[Wednesday 08 September 2010] [08:41:40] <pieterh> it’s always “request-reply”...
[Wednesday 08 September 2010] [08:42:09] <guido_g> ahh... then...
[Wednesday 08 September 2010] [08:42:40] <pieterh> np, easy to fix this in the parser
[Wednesday 08 September 2010] [08:45:52] <pieterh> bah, ditaa does not like Unicode at all, just dies
[Wednesday 08 September 2010] [08:45:53] <pieterh> well, too bad
[Wednesday 08 September 2010] [08:55:20] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [08:55:47] <guido_g> no escape mechanism?
[Wednesday 08 September 2010] [08:57:30] <pieterh> no mechanisms of any kind afaics
[Wednesday 08 September 2010] [08:57:45] <guido_g> oh
[Wednesday 08 September 2010] [08:57:50] <pieterh> which is ok, I’ll email the author
[Wednesday 08 September 2010] [09:16:50] Quit
keffo_ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [09:46:13] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47e87b7 10/ include/zmq.h : EMTHREAD error code returned to zmq.h to ensure backward compatibility -
http://bit.ly/dtznl6[Wednesday 08 September 2010] [10:07:26] <sd88g93> greetings !
[Wednesday 08 September 2010] [10:08:44] <sd88g93> when are the zmq_msg_move() zmq_msg_copy() functions meant to be used ? i mean, what situations call for them ?
[Wednesday 08 September 2010] [10:14:57] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [10:27:50] <cremes> sd88g93: whenever you send data, the library takes ownership of your zmq_msg_t and calls close() on it
[Wednesday 08 September 2010] [10:27:59] <cremes> on the recv side, you are responsible for its lifecycle
[Wednesday 08 September 2010] [10:28:35] <cremes> so you might want to make a copy (which i think does reference counting internally) so that you could send it without it being deallocated
[Wednesday 08 September 2010] [10:28:47] <cremes> the copy is cheap (ref counting)
[Wednesday 08 September 2010] [10:28:56] <cremes> i don’t really know of a use case for move
[Wednesday 08 September 2010] [10:29:14] <sd88g93> cremes: what about VSM’s, where the message is embedded right in the msg struct ?
[Wednesday 08 September 2010] [10:29:23] <cremes> i don’t know
[Wednesday 08 September 2010] [10:29:32] <sd88g93> ohok
[Wednesday 08 September 2010] [10:29:35] <cremes> i’d have to read the code to figure out what happens
[Wednesday 08 September 2010] [10:30:06] <sd88g93> well, what about if you are recieving messages of a multi part message and relaying them through inproc to another thread ?
[Wednesday 08 September 2010] [10:30:19] <sd88g93> is it alright to just recieve and send the same msg ?
[Wednesday 08 September 2010] [10:30:30] <cremes> yes
[Wednesday 08 September 2010] [10:30:48] <cremes> recv does not automatically call close on the message, only sending does
[Wednesday 08 September 2010] [10:30:51] <sd88g93> i’m recieving a multipart message by way of a req/rep socket, and want to relay them to ap ublisher
[Wednesday 08 September 2010] [10:31:32] <sd88g93> but i’m relaying them first through inproc to the main thread, and then forwarding them to the one pub skt
[Wednesday 08 September 2010] [10:31:44] <cremes> oh, so two hops?
[Wednesday 08 September 2010] [10:31:45] <sd88g93> the inproc is failing, and just trying to figure out why
[Wednesday 08 September 2010] [10:31:58] <sd88g93> yeah, 2 hops
[Wednesday 08 September 2010] [10:32:28] <cremes> well, i’m not sure... i haven’t played much with inproc transports
[Wednesday 08 September 2010] [10:32:44] <cremes> as an optimization it might be doing ref counting on the message so that it doesn’t have to copy it for real
[Wednesday 08 September 2010] [10:32:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [10:32:52] <sd88g93> actually, it would be 3 hops, once to the thread, then once to the main thread, and then the forward device forwards them to the final socket for a send
[Wednesday 08 September 2010] [10:33:04] <cremes> only count sockets as hops
[Wednesday 08 September 2010] [10:33:38] <sd88g93> yeah, so 3 hops
[Wednesday 08 September 2010] [10:34:11] <cremes> why don’t you try using copy on the message received that you are sending to the inproc socket and see if that helps (shooting in the dark here)
[Wednesday 08 September 2010] [10:34:36] <sd88g93> yeah, i was trying move
[Wednesday 08 September 2010] [10:34:44] <sd88g93> but maybe copy would be better
[Wednesday 08 September 2010] [10:34:51] <cremes> so, XREQ(tcp) -> XREP(inproc) -> PUB(tcp)
[Wednesday 08 September 2010] [10:34:53] <sd88g93> i dont need to use it after relaying it
[Wednesday 08 September 2010] [10:34:56] <cremes> copy at the XREQ step
[Wednesday 08 September 2010] [10:35:32] <sd88g93> actually, i was using PUB for the inproc
[Wednesday 08 September 2010] [10:35:39] <cremes> the main thing to recall is that whenever you send through a socket, it calls close on the message you just sent
[Wednesday 08 September 2010] [10:35:49] <cremes> you relinquish control of the message to the library whenever you send anything
[Wednesday 08 September 2010] [10:36:13] <cremes> so it sounds to me like when you send to the inproc socket, the message may be getting closed on the send side
[Wednesday 08 September 2010] [10:36:21] <cremes> this is all kind of confusing :)
[Wednesday 08 September 2010] [10:36:33] <cremes> i recommend you ping mato or sustrik
[Wednesday 08 September 2010] [10:37:19] <sd88g93> in the worker thread, it is a PUB skt, and then in the main thread i have a sub skt, so i publish from the thread via inproc, and then it gets fwd’d to the main pub skt
[Wednesday 08 September 2010] [10:37:46] <sd88g93> oh ok
[Wednesday 08 September 2010] [10:37:57] <cremes> are you using a forwarder device or did you roll your own?
[Wednesday 08 September 2010] [10:38:05] <sd88g93> fwd device
[Wednesday 08 September 2010] [10:38:14] <cremes> are you using release 2.0.9?
[Wednesday 08 September 2010] [10:38:25] <sd88g93> yeah, i just updated to 2.0.9
[Wednesday 08 September 2010] [10:38:47] <cremes> ok... i think there was a bug in the forwarder for multi-part message handling that was fixed in 2.0.9
[Wednesday 08 September 2010] [10:39:03] <cremes> that could have been your problem, but if you updated then maybe not
[Wednesday 08 September 2010] [10:39:17] <sd88g93> at first i had my own queue to queue the messages for a publisher thread, that worked, except i’m worried about when i stress test it, if pasing messages allocated on the heap, is a problem when i free them
[Wednesday 08 September 2010] [10:39:46] <sd88g93> yeah, i noticed that in the change log when i updated
[Wednesday 08 September 2010] [10:39:52] <cremes> i’d have to see code to be sure but it doesn’t sound like a problem
[Wednesday 08 September 2010] [10:40:20] <cremes> sorry i couldn’t be of more help... if you figure it out please post a solution
[Wednesday 08 September 2010] [10:40:25] Quit gebi has left this server (Ping timeout: 252 seconds).
[Wednesday 08 September 2010] [10:40:46] <sd88g93> sure, no proble, i realize its still a young project, one of the ramifications
[Wednesday 08 September 2010] [10:41:25] <sd88g93> over all, ive noticed the documentation for zeromq getting much better and more concise as time goes on
[Wednesday 08 September 2010] [10:42:10] <sd88g93> i’ll try using message copy
[Wednesday 08 September 2010] [10:44:03] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [10:44:33] <cremes> great; you might also consider posting your question (with more detail) to the ML
[Wednesday 08 September 2010] [10:52:03] <pieterh> sd88g93: in fact I don’t know what cases need zmq_msg_move and zmq_msg_copy
[Wednesday 08 September 2010] [10:53:10] <sd88g93> ok, thanks, pieterh
[Wednesday 08 September 2010] [10:53:29] <pieterh> perhaps if you want to make abstraction layers that want to grab a copy of a message, work with it, and allow the caller to continue and possibly deallocate it
[Wednesday 08 September 2010] [10:53:48] <pieterh> you certainly do not need them for normal work afaics
[Wednesday 08 September 2010] [10:53:56] <cremes> pieterh: if you want to send the same data to multiple sockets, you need to copy it
[Wednesday 08 September 2010] [10:54:02] <sd88g93> yeah, the move/copy didnt work for my situtaion
[Wednesday 08 September 2010] [10:54:13] <cremes> otherwise your first send will deallocate it out from under you, right?
[Wednesday 08 September 2010] [10:55:03] <mato> cremes: send never deallocates anything out from under you. zmq_msg_t is reference counted.
[Wednesday 08 September 2010] [10:55:35] <cremes> mato: if refcount is 1 and you send the message, it gets deallocated, right?
[Wednesday 08 September 2010] [10:55:40] <sd88g93> oh, so a call to send increments the ref counter
[Wednesday 08 September 2010] [10:55:55] <cremes> i would think it decrements the counter
[Wednesday 08 September 2010] [10:56:03] <pieterh> ... this is all invisible... you send as often as you like and don’t even know there’s a reference counter
[Wednesday 08 September 2010] [10:56:11] <sd88g93> i mean, incrememnt at first, nd then decrement when its done wiht it
[Wednesday 08 September 2010] [10:56:32] <cremes> recv, refcount + 1, send, refcount - 1
[Wednesday 08 September 2010] [10:56:39] <cremes> when 0, dealloc
[Wednesday 08 September 2010] [10:56:44] <mato> cremes: zmq_send() does not touch the refcount
[Wednesday 08 September 2010] [10:56:58] <mato> cremes: you should always do zmq_msg_close () if you want the message to go away
[Wednesday 08 September 2010] [10:56:59] <cremes> doesn’t it call zmq_msg_close?
[Wednesday 08 September 2010] [10:57:12] <mato> not unless something changed; let me check 100%
[Wednesday 08 September 2010] [10:57:16] <cremes> ok
[Wednesday 08 September 2010] [10:57:26] <cremes> i thought i saw that when i reviewed the code a few weeks ago
[Wednesday 08 September 2010] [10:57:35] * pieterh is pretty sure send() does a copy and then a close
[Wednesday 08 September 2010] [10:57:52] <mato> pieterh: no way
[Wednesday 08 September 2010] [10:58:11] <mato> what all the perf tests and examples do is send, then immediately close
[Wednesday 08 September 2010] [10:58:37] <mato> unless i’m confused, check with sustrik to get it from the horses mouth
[Wednesday 08 September 2010] [10:58:41] <pieterh> yes, that’s the calling code
[Wednesday 08 September 2010] [10:58:47] <mato> but this is what i’ve always done in my code
[Wednesday 08 September 2010] [10:58:55] <pieterh> what does the send method do internally to keep the message alive
[Wednesday 08 September 2010] [10:59:02] <pieterh> while the caller does zmq_msg_close?
[Wednesday 08 September 2010] [10:59:08] <mato> ah, sorry
[Wednesday 08 September 2010] [10:59:19] <mato> yeah, send increases the refcount by 1 obviously
[Wednesday 08 September 2010] [10:59:27] <mato> but you get a refcount of 1 already when you create the message
[Wednesday 08 September 2010] [10:59:38] <pieterh> using copy, I assume... ?
[Wednesday 08 September 2010] [10:59:44] <mato> refcount
[Wednesday 08 September 2010] [10:59:49] <mato> what copy?
[Wednesday 08 September 2010] [10:59:59] <sd88g93> oh, so it increases by one, hten when its done sending , it decreases the ref count
[Wednesday 08 September 2010] [11:00:02] <pieterh> so directly modifying message property?
[Wednesday 08 September 2010] [11:00:12] <mato> some lock-free magic with atomic_counter AFAIK
[Wednesday 08 September 2010] [11:00:14] <mato> yes
[Wednesday 08 September 2010] [11:00:31] <mato> all the docs say “zmq_msg_t is opaque”, never touch it outside of the zmq_msg_* functions
[Wednesday 08 September 2010] [11:00:34] <mato> for precisely this reason
[Wednesday 08 September 2010] [11:00:38] <sd88g93> what about with VSM’s ?
[Wednesday 08 September 2010] [11:00:57] <mato> sd88g93: it’s up to 0MQ if it actually copies the content
[Wednesday 08 September 2010] [11:01:18] <pieterh> mato: this is what zmq_msg_copy does, increment refcount
[Wednesday 08 September 2010] [11:01:52] <sd88g93> so, if you have a VSM, and you send it, and then close it, if you re-init the struct, you wipe out what send would have ?
[Wednesday 08 September 2010] [11:02:15] <mato> pieterh: i’m not sure what it actually does internally, why do you need to know?
[Wednesday 08 September 2010] [11:02:19] <sd88g93> as far as i can tell, the VSM is right inside the struct
[Wednesday 08 September 2010] [11:02:23] <mato> pieterh: anyhow, if you need to know, ask sustrik
[Wednesday 08 September 2010] [11:02:27] <pieterh> sure
[Wednesday 08 September 2010] [11:02:35] <mato> sd88g93: ask sustrik on the mailing list, i’m not an expert on what happens internally
[Wednesday 08 September 2010] [11:02:37] <pieterh> the discussion here is “when do we use zmq_msg_copy/move”
[Wednesday 08 September 2010] [11:02:39] <mato> i just know that it works :)
[Wednesday 08 September 2010] [11:02:49] <pieterh> indeed...
[Wednesday 08 September 2010] [11:03:08] <pieterh> i’m just saying, applications do not normally mess with the refcounts, ever
[Wednesday 08 September 2010] [11:03:28] <sd88g93> mato, i’m just having a problem with relaying a message i get from a REP socket over PUB skt using inproc
[Wednesday 08 September 2010] [11:03:29] <pieterh> that’s internal, and i assume that send() calls copy() to grab a reference, and close() when it’s done, to release it
[Wednesday 08 September 2010] [11:03:29] <mato> well, unless you count calling zmq_msg_close() as “messing with the refcount”...
[Wednesday 08 September 2010] [11:03:50] <pieterh> well, sure, but that’s not the semantic
[Wednesday 08 September 2010] [11:03:59] <pieterh> the semantic is “I’m finished with it”, which translates to “decrement refcount”
[Wednesday 08 September 2010] [11:04:04] <mato> yup
[Wednesday 08 September 2010] [11:04:19] * pieterh doesn’t want people to start thinking they have to copy/move messages to send them
[Wednesday 08 September 2010] [11:04:31] <pieterh> sg88g93: the problem is elsewhere
[Wednesday 08 September 2010] [11:04:45] <mato> sd88g93: don’t forget zmq_recv() destroys existing message content, if any
[Wednesday 08 September 2010] [11:05:07] <pieterh> sd88g93, do you have a minimal example?
[Wednesday 08 September 2010] [11:05:28] <sd88g93> i have some code, but so far, i have been unablet to dupliate it in a minimal example,
[Wednesday 08 September 2010] [11:05:47] <mato> sd88g93: so e.g. if you’ve got a loop, doing zmq_msg_init() then zmq_recv(), what could be happening is you recv, then send that message, but then recv back into the same message so you could lose messages
[Wednesday 08 September 2010] [11:05:55] <sd88g93> i posted to the ML one example that i thought i duplicated it , but it didnt
[Wednesday 08 September 2010] [11:06:25] <pieterh> it copies messages from one socket to another, nothing bizarre
[Wednesday 08 September 2010] [11:06:41] <pieterh> if you’re losing messages, perhaps the sockets aren’t connected properly
[Wednesday 08 September 2010] [11:07:22] <mato> sd88g93/pieterh: AFAICT zmq_msg_copy()/zmq_msg_move() are for the case where the app for some reason re-uses message “objects” (i.e. the zmq_msg_t) with a different lifecycle than the message content
[Wednesday 08 September 2010] [11:07:39] <pieterh> right
[Wednesday 08 September 2010] [11:08:06] <mato> sd88g93: what language are you using?
[Wednesday 08 September 2010] [11:08:15] <sd88g93> the c bindings
[Wednesday 08 September 2010] [11:08:57] <pieterh> sd88g93, what is the problem you are actually seeing?
[Wednesday 08 September 2010] [11:09:36] <sd88g93> that’s a minimal case, that one works, but it mirrors what i’m doing in the larger project
[Wednesday 08 September 2010] [11:10:03] <pieterh> so publisher is not sending anything out
[Wednesday 08 September 2010] [11:10:04] <sd88g93> for some reason in the larger project the messages in the main thread arent being recieved via inproc by the subscr socket
[Wednesday 08 September 2010] [11:10:15] <sd88g93> yeah, the publisher isnt sending
[Wednesday 08 September 2010] [11:10:27] <sd88g93> ive isolated it to over the inproc that its failing
[Wednesday 08 September 2010] [11:10:40] <pieterh> try changing the transport to ipc
[Wednesday 08 September 2010] [11:10:55] <sd88g93> when i hcange it to one thread , and publish directly to the bound extreneral interface, it sends fine
[Wednesday 08 September 2010] [11:11:48] <pieterh> so it’s a problem with thread coordination, connections not happening in time
[Wednesday 08 September 2010] [11:11:50] <sd88g93> what’s the difference between inproc and ipc ?
[Wednesday 08 September 2010] [11:12:08] <pieterh> well, inproc requires that you bind first, connect second
[Wednesday 08 September 2010] [11:12:15] <pieterh> ipc like tcp lets you connect first, then bind
[Wednesday 08 September 2010] [11:12:24] <sd88g93> the connections seem to be happending, ive logged it and they seem to happen, the bound channels are made before the connect happens
[Wednesday 08 September 2010] [11:12:36] <pieterh> so with inproc if the subscribers connect before the publisher binds, it won’t work
[Wednesday 08 September 2010] [11:12:42] <pieterh> IMO connect will even return an error
[Wednesday 08 September 2010] [11:13:08] <sd88g93> no, it binds first, and then connects
[Wednesday 08 September 2010] [11:13:35] <sd88g93> that shouldnt be a problem, i think you are correct in that it returns an error, ive recieved that error when that happens
[Wednesday 08 September 2010] [11:13:39] <mato> sd88g93: this is what i meant with my comment about recv’ing into the same message:
http://pastebin.com/tLumsbia[Wednesday 08 September 2010] [11:13:46] <pieterh> yeah, this is your problem I think
[Wednesday 08 September 2010] [11:13:51] <mato> sd88g93: see the difference between the correct/incorrent case
[Wednesday 08 September 2010] [11:13:57] <mato> sd88g93: just in case that’s what you’re doing...
[Wednesday 08 September 2010] [11:14:02] <pieterh> you’re starting the publishers in child threads
[Wednesday 08 September 2010] [11:14:12] <pieterh> but immediately you connect the subscribers to those ports
[Wednesday 08 September 2010] [11:14:15] <pieterh> won’t work
[Wednesday 08 September 2010] [11:14:57] <pieterh> magic solution: do two loops, one to launch pub threads, second to prepare subscribers
[Wednesday 08 September 2010] [11:15:02] <pieterh> do sleep(1) in between two loops
[Wednesday 08 September 2010] [11:15:18] <pieterh> you owe me a beer if it works
[Wednesday 08 September 2010] [11:16:27] <pieterh> here is the text from the user guide:
[Wednesday 08 September 2010] [11:16:27] <pieterh> The {{inproc}} transport has a specific limitation compared to {{ipc}} and {{tcp}}: you must do bind before connect. This is something future versions of 0MQ may fix, but for today it has some impact on the way you use inproc sockets. In the example here, we carefully bind to each endpoint before connecting to it. If you connect first, and then bind, the connect will return an error, and if you ignore that error, the recv will block.
[Wednesday 08 September 2010] [11:17:29] <sd88g93> mato: i dont think that’ sthe case, because i re init the message at the beginning of the loop
[Wednesday 08 September 2010] [11:18:44] <sd88g93> pieterh: it seeems to be binding first before connect
[Wednesday 08 September 2010] [11:18:54] <pieterh> how do you know?
[Wednesday 08 September 2010] [11:19:03] <pieterh> your example does the bind after the connect
[Wednesday 08 September 2010] [11:19:03] <sd88g93> because i print it out
[Wednesday 08 September 2010] [11:19:09] <pieterh> means nothing
[Wednesday 08 September 2010] [11:19:19] <sd88g93> oh
[Wednesday 08 September 2010] [11:19:20] <pieterh> printing from multiple threads can do weird stuff
[Wednesday 08 September 2010] [11:19:35] <pieterh> follow the printf with fflush(stdout) to be sure
[Wednesday 08 September 2010] [11:19:41] <pieterh> sd88g93, next time you have an error, if 0MQ reports an error, that’s kind of the first thing you look at
[Wednesday 08 September 2010] [11:20:16] <pieterh> “I’m using inproc and my connect fails with an error... why?”
[Wednesday 08 September 2010] [11:20:33] <sd88g93> but the connect doesnt fail ,
[Wednesday 08 September 2010] [11:20:33] <
keffo__> is the log pub in 2.0.8?
[Wednesday 08 September 2010] [11:21:27] <pieterh>
keffo__, nope
[Wednesday 08 September 2010] [11:21:33] <pieterh> sd88g93, so what fails?
[Wednesday 08 September 2010] [11:21:46] <pieterh> and could you try the ‘magic fix’ i suggested, it’s a quick test
[Wednesday 08 September 2010] [11:21:57] <sd88g93> the program doesnt throw any errors, it just doesnt publish any of the messages
[Wednesday 08 September 2010] [11:22:08] <sd88g93> yes, i ‘ll tryt that test
[Wednesday 08 September 2010] [11:22:12] <pieterh> ” i think you are correct in that it returns an error, ive recieved that error when that happens”
[Wednesday 08 September 2010] [11:22:15] <pieterh> ?
[Wednesday 08 September 2010] [11:22:35] <sd88g93> sorry, i’m just reading over what everyone has written, its alot to digest
[Wednesday 08 September 2010] [11:22:44] <pieterh> NP :-)
[Wednesday 08 September 2010] [11:22:44] <sd88g93> i’ll try that fix now, thanks,
[Wednesday 08 September 2010] [11:22:58] <pieterh> did you try it with ipc:?
[Wednesday 08 September 2010] [11:23:24] <pieterh> if that works, and inproc does not, it’s most likely the bind/connect order
[Wednesday 08 September 2010] [11:37:17] <sd88g93> no, still nothing
[Wednesday 08 September 2010] [11:37:29] <sd88g93> i’ll try it with ipc , just to be sure
[Wednesday 08 September 2010] [11:37:40] <pieterh> ack
[Wednesday 08 September 2010] [11:39:19] <pieterh> there is some questionable stuff in your example BTW
[Wednesday 08 September 2010] [11:39:32] <pieterh> like threads modifying shared static variables
[Wednesday 08 September 2010] [11:40:39] <sd88g93> yeah, i dont do that n the main program
[Wednesday 08 September 2010] [11:41:34] <sd88g93> the shared static variable is just to get a number to form the pipe number back to the main program, in the actual example i pass it in through the parameter when creating the thread
[Wednesday 08 September 2010] [11:42:18] <pieterh> hang on, do you actually have any subscribers connecting to 127.0.0.1:4000?
[Wednesday 08 September 2010] [11:42:29] <sd88g93> yes
[Wednesday 08 September 2010] [11:43:14] <pieterh> well, what I would do is just break this up into little pieces
[Wednesday 08 September 2010] [11:43:17] <sd88g93> that example does seem to work, but the same scenario, doesnt play out in the real project
[Wednesday 08 September 2010] [11:43:30] <pieterh> ah, it works in the example?
[Wednesday 08 September 2010] [11:43:53] <sd88g93> yeah, just cant duplicate it , that example was just to give you an idea of what i am doing
[Wednesday 08 September 2010] [11:44:10] <pieterh> can’t debug an idea... :-/
[Wednesday 08 September 2010] [11:44:13] <sd88g93> sorry, didnt mean to confuse
[Wednesday 08 September 2010] [11:44:34] <sd88g93> yeah, trying to isolate where its coming from
[Wednesday 08 September 2010] [11:44:50] <pieterh> so you can write your own forwarding code
[Wednesday 08 September 2010] [11:44:55] <pieterh> it is trivial
[Wednesday 08 September 2010] [11:45:02] <pieterh> then you can debug that quite easily
[Wednesday 08 September 2010] [11:45:19] <sd88g93> but when i try a loop in the main program to read what’s coming from the subscriber socket via inproc, nothing comes from it
[Wednesday 08 September 2010] [11:45:31] <pieterh> your problem maybe is that by starting an in-built device you have a black box you can’t inspect
[Wednesday 08 September 2010] [11:45:58] <pieterh> so it’s not the publisher not sending, it’s the subscriber not receiving?
[Wednesday 08 September 2010] [11:46:28] <sd88g93> the publisher isnt making it to the outer publisher socket, nothings coming out
[Wednesday 08 September 2010] [11:46:48] <sd88g93> there’s not errors from the zmq_send function,
[Wednesday 08 September 2010] [11:47:30] <sd88g93> but when i monitor the socket with tshark, there’s nothing coming out
[Wednesday 08 September 2010] [11:47:37] <pieterh> sd88g93, sorry, I can’t really help you any more until/unless you can get a minimal test case that reproduces the problem
[Wednesday 08 September 2010] [11:47:47] Quit
keffo__ has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [11:47:59] <sd88g93> yeah, that’s the problem
[Wednesday 08 September 2010] [12:18:38] Quit
keffo__ has left this server (Ping timeout: 264 seconds).
[Wednesday 08 September 2010] [12:33:04] <cremes> sd88g93: i recommend instrumenting the forwarder device; just have it print what it receives and what it sends out again
[Wednesday 08 September 2010] [12:33:28] <cremes> if you are using multi-part messages with the topic in the first part, then perhaps it isn’t getting forwarded correctly and
[Wednesday 08 September 2010] [12:33:43] <cremes> your subscribers are dropping the messages because the topic doesn’t match
[Wednesday 08 September 2010] [12:43:22] <erickt> are any of the pyzmq devs around today?
[Wednesday 08 September 2010] [12:44:53] Quit
keffo__ has left this server (Ping timeout: 258 seconds).
[Wednesday 08 September 2010] [13:12:48] Join ModusPwnens has joined this channel (
~Slash@75.11.168.223).
[Wednesday 08 September 2010] [13:13:08] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [13:20:07] <ModusPwnens> hmm cremes, yesterday you said that if the performance tests were working correctly, remote_thr should be the first one to exist
[Wednesday 08 September 2010] [13:22:20] <ModusPwnens> exit*
[Wednesday 08 September 2010] [13:22:36] <cremes> right; the publisher should be done before the subscriber
[Wednesday 08 September 2010] [13:22:52] <cremes> but that doesn’t always happen on my machine so i’m a tad confused
[Wednesday 08 September 2010] [13:23:09] <cremes> unfortunately, i don’t have time to look at it right now... maybe someone else can take a peek
[Wednesday 08 September 2010] [13:23:10] <ModusPwnens> Yeah, what I am seeing is the remote continuing to run
[Wednesday 08 September 2010] [13:23:15] <ModusPwnens> until it times out or something
[Wednesday 08 September 2010] [13:24:37] <cremes> i recommend you add some print statements to the tests and run them with low iterations; perhaps something obvious will pop out
[Wednesday 08 September 2010] [13:29:03] <ModusPwnens> ok, will do
[Wednesday 08 September 2010] [13:30:08] Quit gebi has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [14:20:58] Quit
keffo__ has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [14:31:08] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [14:35:55] Quit xla has left this server (Quit: leaving).
[Wednesday 08 September 2010] [14:52:27] Quit
keffo__ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [14:53:02] <mikko> hi
[Wednesday 08 September 2010] [14:55:21] <mikko> what happened with zmq_close semantics?
[Wednesday 08 September 2010] [15:02:30] Quit
keffo__ has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [15:21:20] <mato> mikko: the infrastructure work is all on ‘master’
[Wednesday 08 September 2010] [15:21:36] <mato> mikko: but i’ve not yet had time to test it properly due to “real” work :-(
[Wednesday 08 September 2010] [15:39:25] Quit
keffo__ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [16:20:18] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [16:43:27] Quit
bennymack_ has left this server (Client Quit).
[Wednesday 08 September 2010] [16:45:29] <bennymack> hey everybody! So, i’m looking into 0MQ a bit and so I was strace() a simple example in perl and I noticed a lot of these 24byte send/recv calls on the pipes between the main thread and the 0MQ context thread
[Wednesday 08 September 2010] [16:45:54] <bennymack> is there something I can read about all that chatter that’s going on?
[Wednesday 08 September 2010] [16:46:07] <bennymack> is it the SPB?
[Wednesday 08 September 2010] [17:01:53] Quit GeekGod has left this server (Ping timeout: 276 seconds).
[Wednesday 08 September 2010] [17:01:53] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 08 September 2010] [17:03:33] Quit
GeekGod_ has left this server (Changing host).
[Wednesday 08 September 2010] [17:03:33] Nick
GeekGod_ is now known as GeekGod.
[Wednesday 08 September 2010] [17:09:37] <sd88g93> alhtough i dont think it gets into the low level stuff that youre talking about
[Wednesday 08 September 2010] [17:11:35] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [17:12:45] <mato> bennymack: what you see is signalling between the application thread and zmq I/O thread, see zmq::signaler_t and read the source :-)
[Wednesday 08 September 2010] [17:12:46] <sd88g93> we were talking about it this morning, (or for you , i guess that would be afternoon)
[Wednesday 08 September 2010] [17:13:29] <sd88g93> oh, mato, you were here this morning, i managed to replicate the error i was having in this program :
http://pastebin.com/5Lsy47Cj[Wednesday 08 September 2010] [17:14:12] <mato> sd88g93: hmm, that’s long, do you have a 5-second summary?
[Wednesday 08 September 2010] [17:14:19] <sd88g93> ok,
[Wednesday 08 September 2010] [17:14:45] <sd88g93> the top program is fields requests from a client skt, and feeds them to different threads
[Wednesday 08 September 2010] [17:15:02] <sd88g93> it relays the multipart messages to a pub skt
[Wednesday 08 September 2010] [17:15:19] <sd88g93> (the second program is a client app to send it messages )
[Wednesday 08 September 2010] [17:15:52] <sd88g93> the problem is in the top program, it recives the messages fine , but doesnt relay the messages to the central publisher socket in the main thread
[Wednesday 08 September 2010] [17:16:33] <sd88g93> they should be easy to compile, just gcc -otestpub testpub.c -lzmq for the top program
[Wednesday 08 September 2010] [17:16:36] <mato> sd88g93: so did you find your problem?
[Wednesday 08 September 2010] [17:16:56] <sd88g93> i replicated the same problem in that code
[Wednesday 08 September 2010] [17:17:16] <sd88g93> its in the loop in the top program, (in the thread)
[Wednesday 08 September 2010] [17:17:19] <mato> sorry, it’s kind of late here ...
[Wednesday 08 September 2010] [17:17:25] <sd88g93> oh ok no problem
[Wednesday 08 September 2010] [17:17:26] <bennymack> ok thanks! I’ll take a look
[Wednesday 08 September 2010] [17:17:43] <sd88g93> I can post to the mailing list
[Wednesday 08 September 2010] [17:17:52] <mato> that might be best
[Wednesday 08 September 2010] [17:18:04] <sd88g93> yeah, still 5p.m. here
[Wednesday 08 September 2010] [17:18:09] <jond> sd88g93: is this the same problem as the code on the mail list. that needed a sleep after the pthread_create?
[Wednesday 08 September 2010] [17:18:33] <sd88g93> it might be, not sure which one youre thinking of
[Wednesday 08 September 2010] [17:18:40] <mato> jond: hi Jon... am in the middle of replying to your mail re Git
[Wednesday 08 September 2010] [17:19:16] <mato> jond: is your patmatch branch a private topic branch? i.e. noone pulling from it?
[Wednesday 08 September 2010] [17:19:41] <jond> mato: hi , thanks, also what does ‘Your branch is behind ‘origin/master’ by 86 commits, and can be fast-forwarded.’ mean?
[Wednesday 08 September 2010] [17:20:34] <jond> mato: yes, patmatch private topic branch which I intend to run format patch from and all be hunky dory
[Wednesday 08 September 2010] [17:20:36] <mato> jond: if it’s your local ‘master’ branch that means you can pull from it and you’re 86 commits behind...
[Wednesday 08 September 2010] [17:21:06] <mato> jond: the “can fast forward” stuff basically means you’ve not made any commits to your local master branch that would require a merge
[Wednesday 08 September 2010] [17:21:34] <jond> mato: yep local master; It;s been behind every since I did a hard reset that chuck posted, but i’m not clear how to get rid of it
[Wednesday 08 September 2010] [17:21:54] <mato> jond: git pull
[Wednesday 08 September 2010] [17:22:12] <jond> mato: this is after a git pull
[Wednesday 08 September 2010] [17:22:16] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Wednesday 08 September 2010] [17:23:00] <jond> mato: git pull says this now error: Untracked working tree file ‘src/connect_session.cpp’ would be overwritten by merge.
[Wednesday 08 September 2010] [17:23:22] <mato> jond: oh, right, you have changes you’ve not committed on local master
[Wednesday 08 September 2010] [17:23:28] <mato> jond: what do you want to do with those?
[Wednesday 08 September 2010] [17:24:29] <mato> jond: git status will show you what git thinks about your working tree state
[Wednesday 08 September 2010] [17:24:36] <jond> mato: there shouldnt be any, this all started when I pulled the 2.1 tree then reset to 2.0.8. I havent created those files or edited?
[Wednesday 08 September 2010] [17:24:55] <mato> jond: what does git status say?
[Wednesday 08 September 2010] [17:25:33] <mato> jond: alternatively, do you have any changes you care about in that repository?
[Wednesday 08 September 2010] [17:25:33] <jond> mato: git status gives me the 86 commits behind and some untracked files ; do i need to remove those?
[Wednesday 08 September 2010] [17:27:00] <jond> mato: thats fixed master
[Wednesday 08 September 2010] [17:27:10] <mato> ?
[Wednesday 08 September 2010] [17:27:33] <jond> i removed the ‘overwritten’ files
[Wednesday 08 September 2010] [17:27:48] <mato> you probably wanted git checkout – <path>
[Wednesday 08 September 2010] [17:28:03] <mato> does git status now say “working directory clean” ?
[Wednesday 08 September 2010] [17:29:34] <jond> mato: yes, after I removed some other note files I had left there
[Wednesday 08 September 2010] [17:29:55] <mato> jond: goodo, then you should be able to pull, and follow the instructions in my email
[Wednesday 08 September 2010] [17:30:10] <mato> jond: but if you want to be sure you don’t mess up just take a backup copy of your local repo
[Wednesday 08 September 2010] [17:30:20] <mato> jond: (cp -r zeromq2 zeromq2.bak or something)
[Wednesday 08 September 2010] [17:30:30] <jond> mato: will do , many thanks
[Wednesday 08 September 2010] [17:30:53] <mato> ok, i’m off... cyl
[Wednesday 08 September 2010] [17:31:01] <jond> mato: cyl
[Wednesday 08 September 2010] [17:31:09] <sd88g93> good night , mato
[Wednesday 08 September 2010] [17:31:15] <mato> nite, thx
[Wednesday 08 September 2010] [17:31:16] <sd88g93> ok, just posted to the ML
[Wednesday 08 September 2010] [17:36:42] Part sd88g93 has left this channel (“www.phash.org”).
[Wednesday 08 September 2010] [17:39:00] Quit andrewvc has left this server (Quit: andrewvc).
*** on Wed Sep 8 20:16:49 2010
[Wednesday 08 September 2010] [20:16:49] Topic The channel topic is “Welcome!”.
[Wednesday 08 September 2010] [20:16:55] Mode Channel modes: no messages from outside, topic protection
[Wednesday 08 September 2010] [20:16:55] Created This channel was created on 2010-02-05 10:44.
*** on Thu Sep 9 14:21:58 2010
[Thursday 09 September 2010] [14:21:58] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [14:22:04] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [14:22:04] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [14:30:21] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [14:30:23] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [14:30:24] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [14:30:24] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [14:30:31] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [14:30:31] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [14:30:48] <pieterh> PerfDave, sorry, i was kidding around
[Thursday 09 September 2010] [14:30:53] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [14:31:42] <pieterh> sd88g93, ok, test case is now clear, lemme think about it a sec
*** on Thu Sep 9 15:44:58 2010
[Thursday 09 September 2010] [15:44:58] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [15:45:04] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [15:45:04] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [15:45:13] <pieterh> sd88g93, what is ‘qskt’ for?
[Thursday 09 September 2010] [15:45:31] <sd88g93> that’s the rep socket, in the worker thread, that recieves the messages
[Thursday 09 September 2010] [15:45:35] <sd88g93> from the client
[Thursday 09 September 2010] [15:45:36] <pieterh> there is already a terrible lack of vowels here in Slovakia
[Thursday 09 September 2010] [15:46:06] <sd88g93> tough times, gotta conservee
[Thursday 09 September 2010] [15:46:22] <pieterh> that’s kind of like “who is that guy?” answer: “two legs, body, arms, head”
[Thursday 09 September 2010] [15:47:02] <pieterh> cause it does not in fact talk to the client at all
[Thursday 09 September 2010] [15:47:02] <sd88g93> couldnt come up with a more descriptive variable, sorry, i blame my lack of imagination
[Thursday 09 September 2010] [15:47:19] <sd88g93> it doesnt ? it does on mine
[Thursday 09 September 2010] [15:47:25] <pieterh> it talks to the main thread
[Thursday 09 September 2010] [15:47:26] <pieterh> via inproc
[Thursday 09 September 2010] [15:47:45] <sd88g93> yeah, then there’s a queue device to the main socket
[Thursday 09 September 2010] [15:47:47] <sd88g93> the client skt
[Thursday 09 September 2010] [15:48:11] <pieterh> why is the client not talking to the worker directly?
[Thursday 09 September 2010] [15:48:28] <sd88g93> if you have more than one thread
[Thursday 09 September 2010] [15:48:44] <sd88g93> use the device , at least that’s the scenario
[Thursday 09 September 2010] [15:48:55] <pieterh> well...
[Thursday 09 September 2010] [15:49:00] <pieterh> it
[Thursday 09 September 2010] [15:49:15] <pieterh> the device is really when you have an N-to-N case
[Thursday 09 September 2010] [15:49:23] <pieterh> and don’t want all the N’s to have to know about each other
[Thursday 09 September 2010] [15:49:43] <sd88g93> yeah, there’s just one client now, but there could be more
[Thursday 09 September 2010] [15:49:46] <pieterh> ah... you are in fact developing the MT server model...
[Thursday 09 September 2010] [15:49:54] <sd88g93> yeah
[Thursday 09 September 2010] [15:49:57] <pieterh> i hardly recognized it... :-/
[Thursday 09 September 2010] [15:49:59] <sd88g93> there could be more than one client
[Thursday 09 September 2010] [15:50:58] <sd88g93> well, a diffferent form of it, i guess
[Thursday 09 September 2010] [15:53:00] <sd88g93> zhelpers.h:25 i thought sprintf was “evil”, suppose to use snprintf ?
[Thursday 09 September 2010] [15:54:16] <keffo> pieterh, I’ve done the lru now!
[Thursday 09 September 2010] [15:54:34] <pieterh> keffo: wow!
[Thursday 09 September 2010] [15:54:41] <pieterh> sd88g93, this is what the worker loop becomes:
[Thursday 09 September 2010] [15:54:44] <pieterh> char
*command = s_recv (control);
[Thursday 09 September 2010] [15:54:44] <pieterh> char
*count = s_recv (control);
[Thursday 09 September 2010] [15:54:44] <pieterh> puts (“Worker got command: %sn”, command);
[Thursday 09 September 2010] [15:54:44] <pieterh> s_send (publisher, “msg.x|”);
[Thursday 09 September 2010] [15:54:47] <pieterh> s_sendmore (publisher, command);
[Thursday 09 September 2010] [15:54:47] <pieterh> s_send (publisher, count);
[Thursday 09 September 2010] [15:54:48] <pieterh> free (command);
[Thursday 09 September 2010] [15:54:50] <pieterh> free (count);
[Thursday 09 September 2010] [15:54:52] <pieterh> s_send (control, “OK”);
[Thursday 09 September 2010] [15:54:59] <pieterh> there is a bug, i see immediately...
[Thursday 09 September 2010] [15:55:03] <pieterh> in my code, not yours
[Thursday 09 September 2010] [15:55:24] <ModusPwnens> Wait a sec pieter, i was referring to something on the site
[Thursday 09 September 2010] [15:55:49] <sd88g93> oh, in s_send and s_sendmore you do init the message and close it each time,
[Thursday 09 September 2010] [15:56:16] <sd88g93> yeah, that’s what i thought, you were just referrring to coding style, eh ? not actuall instructions needed
[Thursday 09 September 2010] [15:56:24] <sd88g93> i get ya now, ok
[Thursday 09 September 2010] [15:56:50] <pieterh> ModusPwnens, even better, join the site and click ‘edit’
[Thursday 09 September 2010] [15:56:57] <pieterh> it’s a wiki :-)
[Thursday 09 September 2010] [15:57:07] <ModusPwnens> Oh okay, well that’s easy to fix then
[Thursday 09 September 2010] [15:57:09] <keffo> pieterh, it seems to be working very well.. I just need to post back the results now, but I’ll probably use a new socketlayer for that, with some persistance as well
[Thursday 09 September 2010] [15:57:25] <pieterh> hmm, nice
[Thursday 09 September 2010] [15:57:38] <pieterh> keffo: i still find working with envelopes to be too painful
[Thursday 09 September 2010] [15:57:52] <pieterh> but we’ll make abstractions for those later
[Thursday 09 September 2010] [15:58:47] <keffo> I have a Route class for that
[Thursday 09 September 2010] [15:59:02] <pieterh> yeah, right, you told me... nice
[Thursday 09 September 2010] [15:59:04] <keffo> which can construct itself from incoming, and also send it
[Thursday 09 September 2010] [15:59:07] <keffo> yeah
[Thursday 09 September 2010] [15:59:19] <pieterh> were the names in the routing section too weird?
[Thursday 09 September 2010] [15:59:27] <pieterh> i was trying to get mnemonics that worked
[Thursday 09 September 2010] [15:59:27] <keffo> where?
[Thursday 09 September 2010] [15:59:31] <pieterh> ch3
[Thursday 09 September 2010] [15:59:32] <keffo> mama,papa? =)
[Thursday 09 September 2010] [15:59:41] <pieterh> yeah
[Thursday 09 September 2010] [15:59:49] <jhawk28> the mama and papa were hilarious
[Thursday 09 September 2010] [15:59:52] <keffo> papa was a bit confusing, didnt really see the application to be honest, but I was mostly interested in the mama anyway :)
[Thursday 09 September 2010] [16:00:14] <pieterh> well, good, it’s hard to make this material even remotely funny
[Thursday 09 September 2010] [16:00:25] * pieterh still has to write a short piece on quick and clean exits
[Thursday 09 September 2010] [16:00:36] <jhawk28> we enjoyed it at work
[Thursday 09 September 2010] [16:00:37] <psino> pieterh: you should move some more reasoning about the naming in ch3 to the place where you introduce them
[Thursday 09 September 2010] [16:00:46] <keffo> pieterh, Another interesting sideffect of the mama topology you presented there is that there is no longer any immediate necessity for a heartbeat system
[Thursday 09 September 2010] [16:00:53] <pieterh> psino: did I get the order wrong?
[Thursday 09 September 2010] [16:01:07] <pieterh> yup, but you need heartbeats for another reason
[Thursday 09 September 2010] [16:01:26] <pieterh> i was going to write that today but got distracted by our friend sd88g93’s curious case here
[Thursday 09 September 2010] [16:01:36] <pieterh> also, ISP guys getting lost
[Thursday 09 September 2010] [16:01:38] <keffo> sure, but not in the same way as a topdown design
[Thursday 09 September 2010] [16:02:03] <psino> no, but when you say that REQ is a MAMA socket, making insistent demands, you might want to add something more that helps the reader remember the terminology
[Thursday 09 September 2010] [16:02:26] <pieterh> hmm, lemme think about that for a second...
[Thursday 09 September 2010] [16:02:52] <pieterh> i wrote that text very late on tuesday evening after a glass or two of wine... :-/
[Thursday 09 September 2010] [16:03:05] <psino> you describe the usage of mama pretty good in the “Router-to-Mama”-section
[Thursday 09 September 2010] [16:03:23] <psino> (personal opinion of course)
[Thursday 09 September 2010] [16:03:30] <pieterh> i think the character is based on my friend Fabio’ Italian mother
[Thursday 09 September 2010] [16:03:36] <pieterh> ok...
[Thursday 09 September 2010] [16:03:44] <pieterh> opinion is better than silence
[Thursday 09 September 2010] [16:04:33] <jhawk28> I was going through the swap and hwm parts of the user guide and I got an exception after the pub ran for a while (from yquery:108)
[Thursday 09 September 2010] [16:05:13] <pieterh> jhawk28, interesting!
[Thursday 09 September 2010] [16:05:23] <jhawk28> It died when I was using the connect, but not the bind
[Thursday 09 September 2010] [16:06:03] <jhawk28> not sure if it matters, but I was using the Java bindings
[Thursday 09 September 2010] [16:06:14] <pieterh> do you mean yqueue?
[Thursday 09 September 2010] [16:06:30] <jhawk28> yes, it was an assertion that failed
[Thursday 09 September 2010] [16:06:52] <jhawk28> going from memory - so it could be wrong
[Thursday 09 September 2010] [16:06:58] <pieterh> could you please log this on the issue tracker, along with the hardware setup you’re using?
[Thursday 09 September 2010] [16:07:12] <jhawk28> I think so
[Thursday 09 September 2010] [16:07:12] <pieterh> this should absolutely never never happen
[Thursday 09 September 2010] [16:07:27] <jhawk28> figured as much
[Thursday 09 September 2010] [16:07:30] <pieterh> there is indeed an assertion on yqueue.hpp:108
[Thursday 09 September 2010] [16:07:42] <pieterh> woa, hang on
[Thursday 09 September 2010] [16:07:49] <pieterh> it’s an out of memory condition
[Thursday 09 September 2010] [16:08:36] <jhawk28> hwm was set to 1 and swap was set to 25000000
[Thursday 09 September 2010] [16:09:27] <pieterh> you sure you didn’t reverse those? :-)
[Thursday 09 September 2010] [16:09:54] <pieterh> it’s worth logging, we’ll run that on some big boxes here and see if we can reproduce it
[Thursday 09 September 2010] [16:09:56] <jhawk28> :) was following the guide and then started playing around seeing what it would do
[Thursday 09 September 2010] [16:09:59] <pieterh> thanks
[Thursday 09 September 2010] [16:14:28] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [16:14:29] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [16:14:30] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [16:14:31] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [16:14:37] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [16:14:37] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [16:14:48] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [16:15:06] <daggertip> so
[Thursday 09 September 2010] [16:16:15] Part daggertip has left this channel.
[Thursday 09 September 2010] [16:17:16] * pieterh salutes the short-lived but nicely named daggertip
[Thursday 09 September 2010] [16:17:36] * cremes ha
[Thursday 09 September 2010] [16:23:10] <pieterh> sd88g93, you still there? I award you the “obfuscated 0MQ contest” award :-)
[Thursday 09 September 2010] [16:24:33] <pieterh> you have six sockets and two devices in your minimal test case
[Thursday 09 September 2010] [16:45:19] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [16:45:20] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [16:45:21] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [16:45:27] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [16:45:27] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [16:46:41] <sd88g93> i’m always up for an award
[Thursday 09 September 2010] [16:47:41] <sd88g93> think sustrik can have a looksie ?
[Thursday 09 September 2010] [16:54:15] <pieterh> sd88g93, lol, are you trolling me?
[Thursday 09 September 2010] [16:54:34] <sd88g93> trolling ?
[Thursday 09 September 2010] [16:54:36] <pieterh> i’ve stripped it right down and removing the queue device lets it work
[Thursday 09 September 2010] [16:54:39] <pieterh> :-)
[Thursday 09 September 2010] [16:54:56] <pieterh> gnash, i will find this... gnash
[Thursday 09 September 2010] [16:55:14] <sd88g93> how do you read the messages into the thread ?
[Thursday 09 September 2010] [16:55:28] <pieterh> well, connect worker directly to client
[Thursday 09 September 2010] [16:55:47] <pieterh> next time, develop such code step by step so you can test it gradually
[Thursday 09 September 2010] [16:56:03] <pieterh> and when you add a new piece if it does not work you know exactly what is questionable
[Thursday 09 September 2010] [16:56:07] <sd88g93> i’m suspicious that the messages are being altered in some way, like martin is slipping coded messages in there
[Thursday 09 September 2010] [16:56:18] <sd88g93> yeah, that’s what i did , but ....
[Thursday 09 September 2010] [16:56:24] <pieterh> i doubt it
[Thursday 09 September 2010] [16:56:29] <sd88g93> I realized zmq can do more
[Thursday 09 September 2010] [16:57:10] <sd88g93> I had a little different setup before, and then i put the forwarder back in , because i really didnt want to move zmq_msg_t ‘s around between threads
[Thursday 09 September 2010] [16:57:24] <pieterh> that is a messed up version but it shows you what it looks like
[Thursday 09 September 2010] [16:57:47] <pieterh> see how the queue thread creates its own sockets
[Thursday 09 September 2010] [16:58:06] <pieterh> not necessary to do that in main, kind of pointless and... in fact... illegal
[Thursday 09 September 2010] [16:58:15] <pieterh> you cannot create socket in thread A and use it in thread B
[Thursday 09 September 2010] [17:01:47] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [17:01:49] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [17:01:50] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:01:50] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:01:57] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:01:57] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:02:10] Quit
travlr_ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:05:13] <pieterh> ok, sd88g93 that’s the problem
[Thursday 09 September 2010] [17:05:30] <sd88g93> oh really
[Thursday 09 September 2010] [17:06:10] <pieterh> dude, you owe me a case of beer
[Thursday 09 September 2010] [17:06:36] <sd88g93> yes yes, i do hahaa
[Thursday 09 September 2010] [17:06:48] <pieterh> the publisher socket is not there, you can put it back of course
[Thursday 09 September 2010] [17:07:07] <pieterh> so problem was that your queue thread wasn’t working reliably
[Thursday 09 September 2010] [17:07:21] <pieterh> when you test code, don’t run it once, run it 10-100 times
[Thursday 09 September 2010] [17:07:31] <sd88g93> so you just create the skts for the device in that thread then,
[Thursday 09 September 2010] [17:07:35] <pieterh> threading issues are nasty like that
[Thursday 09 September 2010] [17:07:47] <pieterh> pls dnt sy skts t mks m brn hrt
[Thursday 09 September 2010] [17:08:01] <pieterh> socks if you really have to save on letters
[Thursday 09 September 2010] [17:08:12] <pieterh> but yes, you do
[Thursday 09 September 2010] [17:08:15] <sd88g93> what’s wrong with skts ?
[Thursday 09 September 2010] [17:08:32] <pieterh> what’s wrong with the letters ‘o’ and ‘e’?
[Thursday 09 September 2010] [17:08:33] <pieterh> :-)
[Thursday 09 September 2010] [17:08:52] <pieterh> seriously, if you remove essential letters from words you create extra parsing costs for the reader
[Thursday 09 September 2010] [17:08:59] <sd88g93> sorry, that’s just how i do variables, i tend to leave out vowels
[Thursday 09 September 2010] [17:09:08] <pieterh> english may look redundant but it’s been elegantly evolved for lazy readers
[Thursday 09 September 2010] [17:09:25] <pieterh> if you want a hint for style: make your code readable
[Thursday 09 September 2010] [17:09:37] <pieterh> please compare my versions with yours
[Thursday 09 September 2010] [17:09:58] <sd88g93> those zhelper.h are helpful, but unfortunately, i need it for binary data
[Thursday 09 September 2010] [17:10:01] <pieterh> you seriously cost me several hours just to untangle your code...
[Thursday 09 September 2010] [17:10:16] <pieterh> zhelpers.h is an example, modify/extend it for binary data
[Thursday 09 September 2010] [17:10:20] Quit psino has left this server (Quit: psino).
[Thursday 09 September 2010] [17:10:23] <pieterh> s_recv already works for binary data
[Thursday 09 September 2010] [17:10:30] <sd88g93> well, i was hoping for zero copy as well
[Thursday 09 September 2010] [17:10:32] <pieterh> add s_send_bin (socket, data, size) and it’s done
[Thursday 09 September 2010] [17:10:48] <sd88g93> ok,
[Thursday 09 September 2010] [17:10:58] <pieterh> just make sure to copy the data
[Thursday 09 September 2010] [17:11:04] <pieterh> otherwise you’ll get weirdness too
[Thursday 09 September 2010] [17:11:44] <pieterh> unless you are doing a lot of traffic (100K+ / second) or very large messages, zerocopy won’t be an advantage
[Thursday 09 September 2010] [17:11:52] <pieterh> but it will make your code harder to write and read and fix
[Thursday 09 September 2010] [17:11:53] <sd88g93> well, there’s no way to do zero copy though, when i can find a way to do zerocopy and those functions , then we’ve got something
[Thursday 09 September 2010] [17:12:09] <pieterh> do not optimize code that isn’t working
[Thursday 09 September 2010] [17:12:17] <pieterh> it is really a bad idea
[Thursday 09 September 2010] [17:12:31] <pieterh> actually I need to totally remove all reference to zerocopy from the user guide
[Thursday 09 September 2010] [17:12:37] <sd88g93> well, one message part can be pretty big
[Thursday 09 September 2010] [17:12:41] <pieterh> people should not even know it exists until chapter 4 or so...
[Thursday 09 September 2010] [17:12:47] <pieterh> doesn’t matter
[Thursday 09 September 2010] [17:12:55] <sd88g93> like 100k
[Thursday 09 September 2010] [17:13:02] <pieterh> doesn’t matter
[Thursday 09 September 2010] [17:13:03] <sd88g93> 100KB , easily
[Thursday 09 September 2010] [17:13:07] <pieterh> seriously, do the test yourself
[Thursday 09 September 2010] [17:13:21] <sd88g93> really ? it wont matter ?
[Thursday 09 September 2010] [17:13:36] <pieterh> for (namewithvowels = 0; nwv < 1000; nwv++) { do large copy here }
[Thursday 09 September 2010] [17:13:39] <sd88g93> hmm ok , i’ll have to think on that some
[Thursday 09 September 2010] [17:13:41] <pieterh> run, test, prove it to yourself
[Thursday 09 September 2010] [17:13:58] <pieterh> it matters if you are into microsecond latency issues and CPUs are running at 75% full
[Thursday 09 September 2010] [17:14:11] <pieterh> i’ll bet you that case of beer x 2 that’s not your case
[Thursday 09 September 2010] [17:14:41] <pieterh> premature optimization is one of the many roads to evil and it’s not even a particularly fun one
[Thursday 09 September 2010] [17:15:07] <gebi> zero-copy can also easily be more expensive than plain copy if not aligned on cache-lines and between different cpus
[Thursday 09 September 2010] [17:18:26] <pieterh> ok, folk, /me has to go sleepy byes
[Thursday 09 September 2010] [17:22:16] Nick Nickname already in use. Trying travlr.
[Thursday 09 September 2010] [17:22:17] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Thursday 09 September 2010] [17:22:17] Notice -NickServ- You are now identified for travlr.
[Thursday 09 September 2010] [17:22:18] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:22:19] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:22:25] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:22:25] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:22:57] Quit
travlr__ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:23:26] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:24:28] Quit sd88g93 has left this server (Quit: Ex-Chat).
[Thursday 09 September 2010] [17:31:42] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [17:31:44] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [17:31:44] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:31:45] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:31:48] Quit travlr has left this server (Ping timeout: 245 seconds).
[Thursday 09 September 2010] [17:31:52] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:31:52] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:55:12] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [17:55:14] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [17:55:15] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:55:15] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:55:22] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:55:22] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:56:18] Quit
travlr_ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [18:40:07] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [18:40:08] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [18:40:09] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [18:40:15] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [18:40:15] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [18:42:04] Quit kelvink has left this server (Quit: Leaving.).
[Thursday 09 September 2010] [18:59:09] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:04:37] Nick Nickname already in use. Trying travlr.
[Thursday 09 September 2010] [19:04:39] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Thursday 09 September 2010] [19:04:39] Notice -NickServ- You are now identified for travlr.
[Thursday 09 September 2010] [19:04:39] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:04:40] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:04:47] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:04:47] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:04:56] Quit
travlr__ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:15:04] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [19:15:06] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [19:15:06] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:15:07] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:15:14] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:15:14] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:15:34] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:25:31] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [19:25:33] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [19:25:33] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:25:34] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:25:40] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:25:41] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:26:23] Quit
travlr_ has left this server (Ping timeout: 276 seconds).
*** on Thu Sep 9 20:09:53 2010
[Thursday 09 September 2010] [20:09:53] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [20:09:59] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [20:09:59] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [20:18:18] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [20:18:24] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [20:18:24] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [20:19:11] Quit travlr has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 11 03:24:23 2010
[Saturday 11 September 2010] [03:24:23] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [03:24:29] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [03:24:29] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [03:31:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [03:53:07] Quit ivan has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [04:08:05] Quit rgl has left this server (Ping timeout: 272 seconds).
[Saturday 11 September 2010] [04:16:31] Join gebi has joined this channel (
~gebi@84.119.81.11).
[Saturday 11 September 2010] [05:22:59] Quit ivan has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 11 06:07:38 2010
[Saturday 11 September 2010] [06:07:38] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [06:07:44] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [06:07:44] Created This channel was created on 2010-02-05 10:44.
*** on Sat Sep 11 06:32:47 2010
[Saturday 11 September 2010] [06:32:47] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [06:32:53] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [06:32:53] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [07:44:15] <icy> I can connect a socket to many endpoints, what is the reason for not allowing one to disconnect from an endpoint again?
[Saturday 11 September 2010] [08:00:29] <icy> also why do I have to loop over all sockets to see which ones got events after a zmq_poll(), I can’t see how this would scale
[Saturday 11 September 2010] [08:03:36] <guido_g> that’s how poll works
[Saturday 11 September 2010] [08:08:57] <icy> well that kinda nullifies a benefit that for example epoll() gives you
[Saturday 11 September 2010] [08:09:05] <icy> where you don’t have to loop around searching
[Saturday 11 September 2010] [08:11:16] <guido_g> a) somewhere you have to go through the list of fds
[Saturday 11 September 2010] [08:12:11] <guido_g> b) mq sockets aggregate os level sockets (one mq socket -> many os level sockets), so you have only a few mq sockets
[Saturday 11 September 2010] [08:12:37] <guido_g> c) mq uses epoll or whatever is available on the given platform
[Saturday 11 September 2010] [08:12:42] <icy> yea but due to not being able to disconnect from one endpoint, I might end up needing many zmq sockets
[Saturday 11 September 2010] [08:13:07] <guido_g> i don’t dare to ask why...
[Saturday 11 September 2010] [08:13:08] <icy> it uses epoll but makes me loop over all sockets, epoll does not require me to do that
[Saturday 11 September 2010] [08:13:45] <guido_g> epoll is not available on all platforms mq is running on
[Saturday 11 September 2010] [08:13:49] <icy> I know
[Saturday 11 September 2010] [08:13:52] <guido_g> so?
[Saturday 11 September 2010] [08:14:02] <icy> but many provide smarter facilities than poll()
[Saturday 11 September 2010] [08:14:44] <icy> so you throw away a lot of benefits for a lowest common denominator
[Saturday 11 September 2010] [08:15:03] <guido_g> so you think it would make sense to have mq behave differently on differnt platforms?
[Saturday 11 September 2010] [08:15:17] <icy> that is not what I am saying
[Saturday 11 September 2010] [08:15:21] <guido_g> it is
[Saturday 11 September 2010] [08:15:27] <icy> nope
[Saturday 11 September 2010] [08:15:55] <guido_g> because if you use the _best_ way on a platform, you also have the platform specific semantics
[Saturday 11 September 2010] [08:16:15] <icy> you can still abstract it in a general way
[Saturday 11 September 2010] [08:16:24] <icy> see all event libraries like libev or libevent
[Saturday 11 September 2010] [08:16:56] <gebi> icy: just for the record, epoll is NOT faster than poll in some situations, quite frankly it is 20% or more slower than poll in some situations
[Saturday 11 September 2010] [08:17:11] <icy> I know, for small sets poll() is nice
[Saturday 11 September 2010] [08:17:35] <guido_g> may be these abstraction would cost a lot on some platforms, so why punish people not running you favorite platform?
[Saturday 11 September 2010] [08:18:05] <icy> they do not cost any considerable amount
[Saturday 11 September 2010] [08:18:46] <guido_g> evidence by allegation?
[Saturday 11 September 2010] [08:19:01] <icy> what evidence do you provide?
[Saturday 11 September 2010] [08:19:48] <guido_g> mq is built that way, and works
[Saturday 11 September 2010] [08:21:32] <icy> well all I’m saying is I cannot see how it can scale to many zmq sockets because of all the looping. so clearly there must be a reason for that and that would that you would not need many zqm sockets as each can connect to many transport endpoints
[Saturday 11 September 2010] [08:22:07] <icy> which brings me to my initial question why you can’t disconnect from one again
[Saturday 11 September 2010] [08:23:15] <icy> I might be missing something in how you are supposed to handle many volatile connections and am ready for enlightenment :)
[Saturday 11 September 2010] [08:25:11] <guido_g> what kind of connections?
[Saturday 11 September 2010] [08:25:32] <guido_g> you don’t see connects at the mq level
[Saturday 11 September 2010] [08:28:26] <icy> hm I was thinking of a socket which connects to many IPs over tcp but I guess I just need to flip it around and make it the server and not client
[Saturday 11 September 2010] [08:29:25] <guido_g> even then you would have only one mq socket
[Saturday 11 September 2010] [08:29:33] <pieter_hintjens> gebi: yes, i recall Zed Shaw’s analysis, quite fun
[Saturday 11 September 2010] [08:30:03] <pieter_hintjens> icy: you’re IMO trying to apply existing TCP patterns to 0MQ
[Saturday 11 September 2010] [08:30:03] <icy> yea but if the IPs are dynamic, the zmq socket would just get more and more endpoints attached to it
[Saturday 11 September 2010] [08:30:17] <icy> pieter_hintjens: quite possible, I am used to that :)
[Saturday 11 September 2010] [08:30:55] <guido_g> if you connect more and more sockets to a remote socket, then yes, the number of connections is growing
[Saturday 11 September 2010] [08:30:57] <pieter_hintjens> icy: you need to think of 0MQ sockets as mini applications not just APIs
[Saturday 11 September 2010] [08:31:08] <pieter_hintjens> they are intelligent and do things for you
[Saturday 11 September 2010] [08:31:24] <pieter_hintjens> learning this is the key to using 0MQ right and getting it to scale and be useful for you
[Saturday 11 September 2010] [08:31:54] <pieter_hintjens> there are some generic rules that apply to all 0MQ sockets
[Saturday 11 September 2010] [08:32:02] <pieter_hintjens> and then there are specifics to each type, how they behave
[Saturday 11 September 2010] [08:32:12] <pieter_hintjens> generically, client vs. server is important to get right
[Saturday 11 September 2010] [08:32:35] <pieter_hintjens> stable nodes act better as servers (doing the bind), dynamic nodes as clients (doing the connect)
[Saturday 11 September 2010] [08:32:36] <icy> yup makes sense, I think it will just take a bit of time to get the old patterns out of my head
[Saturday 11 September 2010] [08:32:39] <pieter_hintjens> yup
[Saturday 11 September 2010] [08:32:50] <pieter_hintjens> if it’s any help, we all go through this
[Saturday 11 September 2010] [08:32:59] <pieter_hintjens> did you read the user guide and try the examples?
[Saturday 11 September 2010] [08:33:05] <icy> yes
[Saturday 11 September 2010] [08:33:16] <pieter_hintjens> did you just read it, or try the code?
[Saturday 11 September 2010] [08:33:36] <icy> I tried not all but some
[Saturday 11 September 2010] [08:33:38] <icy> and read all
[Saturday 11 September 2010] [08:33:44] <pieter_hintjens> well, it takes time too
[Saturday 11 September 2010] [08:33:58] <pieter_hintjens> several days of slow reprogramming
[Saturday 11 September 2010] [08:34:02] <guido_g> it needs time to sink in
[Saturday 11 September 2010] [08:34:15] <guido_g> mq can’t be groked in a day, imho
[Saturday 11 September 2010] [08:34:46] <pieter_hintjens> if you never did networking programming you’d have less trouble unlearning old patterns
[Saturday 11 September 2010] [08:34:51] <guido_g> the things that makes it so good is what makes it so hard :)
[Saturday 11 September 2010] [08:34:53] <pieter_hintjens> but you’d still have to learn the new stuff
[Saturday 11 September 2010] [08:35:09] <guido_g> but who would that be?
[Saturday 11 September 2010] [08:35:11] <pieter_hintjens> icy: briefly, you trust 0MQ sockets to do a lot for you
[Saturday 11 September 2010] [08:35:34] <pieter_hintjens> there is a lot less of the command-and-control you need with traditional sockets
[Saturday 11 September 2010] [08:36:32] <icy> well I understand how stuff works, not very complicated after all. I’m trying now to understand why the api is how it is. e.g. normally event libs let you specify callbacks that get called upon events (hence I was so surprised by the looping needed after zmq_poll)
[Saturday 11 September 2010] [08:36:51] <pieter_hintjens> icy: the api aims to look like plain boring sockets
[Saturday 11 September 2010] [08:37:05] <pieter_hintjens> the poll stuff is not very nice
[Saturday 11 September 2010] [08:37:13] <pieter_hintjens> and we’ll probably make it nicer for 3.0
[Saturday 11 September 2010] [08:37:19] <icy> fair enough
[Saturday 11 September 2010] [08:37:37] <pieter_hintjens> there is a moral when you make new technology
[Saturday 11 September 2010] [08:37:43] <pieter_hintjens> if it looks new, people won’t adopt it
[Saturday 11 September 2010] [08:38:16] <guido_g> hmmm
[Saturday 11 September 2010] [08:38:42] <pieter_hintjens> people = except pioneers who have a fetish for new technology just on principle
[Saturday 11 September 2010] [08:38:54] <pieter_hintjens> and who will abandon it for the next new thing without hesitation
[Saturday 11 September 2010] [08:39:26] <guido_g> i hmmed because the last 30 years have seen a lot of new things
[Saturday 11 September 2010] [08:39:40] <pieter_hintjens> I read a lovely quote from the inventor of liquid salt nuclear reactors
[Saturday 11 September 2010] [08:39:54] <pieter_hintjens> sure... i’m making a horrid generalization
[Saturday 11 September 2010] [08:39:59] <guido_g> ack
[Saturday 11 September 2010] [08:39:59] <icy> I’m not so sure if this can be applied so easily
[Saturday 11 September 2010] [08:40:02] <guido_g> go on :)
[Saturday 11 September 2010] [08:40:11] <pieter_hintjens> “Perhaps the moral to be drawn is that a technology that differs too much from an existing technology has not one hurdle to overcome–to demostrate its feasibility–but another even greater one–to convince influential individuals and organizations who are intellectually and emotionally attached to a different technology that they should adopt the new path”
[Saturday 11 September 2010] [08:40:29] <guido_g> icy: it’s a trade-off between usability and speed, mostly
[Saturday 11 September 2010] [08:40:30] <icy> if it was true, no radically things would have been adopted in the past but there have been plenty :)
[Saturday 11 September 2010] [08:40:50] <pieter_hintjens> ah, (a) there are always exceptions to a general rule and (b) there are ways to appear old yet introduce newness
[Saturday 11 September 2010] [08:40:56] <guido_g> i’ve seen a messaging lib w/ callbacks, and it wasn’t fun at all
[Saturday 11 September 2010] [08:41:20] <icy> in my opinion anything providing a considerable advantage over something else has good chances of getting adopted
[Saturday 11 September 2010] [08:41:36] <icy> provided it does not have disadvantages that outweight the advantages of course :P
[Saturday 11 September 2010] [08:41:44] <guido_g> you remember the video-tape thingy?
[Saturday 11 September 2010] [08:41:46] <pieter_hintjens> this is the engineer point of view but it ignores people emotional attachment to stuff they know
[Saturday 11 September 2010] [08:41:52] <pieter_hintjens>
*people’s
[Saturday 11 September 2010] [08:41:57] <pieter_hintjens>
*peoples’
[Saturday 11 September 2010] [08:42:06] <pieter_hintjens> sms text messages
[Saturday 11 September 2010] [08:42:10] <pieter_hintjens> email
[Saturday 11 September 2010] [08:42:10] <icy> guido_g: what about fire, electricity etc? :)
[Saturday 11 September 2010] [08:42:22] <pieter_hintjens> icy: fire has callbacks?
[Saturday 11 September 2010] [08:42:30] <icy> there are examples for both sides
[Saturday 11 September 2010] [08:42:35] <icy> pieter_hintjens: uhm.. ;)
[Saturday 11 September 2010] [08:42:40] <pieter_hintjens> :-)
[Saturday 11 September 2010] [08:43:03] <guido_g> icy: but it doesn’t work that way
[Saturday 11 September 2010] [08:43:10] <pieter_hintjens> anyhow, this is why 0MQ very carefully emulates a poll function everyone hates but totally trusts and understands
[Saturday 11 September 2010] [08:44:12] <icy> guido_g: just curious, why was that lib no fun?
[Saturday 11 September 2010] [08:44:25] <guido_g> icy: wllm
[Saturday 11 September 2010] [08:44:39] <icy> pieter_hintjens: I totally see your point
[Saturday 11 September 2010] [08:45:02] <icy> wllm?
[Saturday 11 September 2010] [08:45:09] <pieter_hintjens> icy: once there is a significant weight behind 0MQ (and we’re getting there) we can be more ambitious about our own abstractions
[Saturday 11 September 2010] [08:45:41] <guido_g> icy: from a small 3 letter company (HAL) :)
[Saturday 11 September 2010] [08:46:12] <icy> pieter_hintjens: wouldn’t it be hard to make a radical api change once you got a considerable amount of applications using it? as a counter argument
[Saturday 11 September 2010] [08:46:17] <icy> guido_g: will take a look, thx
[Saturday 11 September 2010] [08:46:21] <pieter_hintjens> icy: it’s about layers
[Saturday 11 September 2010] [08:46:28] <guido_g> icy: no need to, really :)
[Saturday 11 September 2010] [08:46:31] <pieter_hintjens> we’d never make radical api changes
[Saturday 11 September 2010] [08:46:41] <pieter_hintjens> just add layers on top
[Saturday 11 September 2010] [08:47:14] <pieter_hintjens> e.g. event driven frameworks that actually call the poll API
[Saturday 11 September 2010] [08:47:32] <pieter_hintjens> though I suspect poll API itself will be redesigned before the 0MQ is totally stabilized
[Saturday 11 September 2010] [08:47:40] <guido_g> so the developer can pick his poison depending on the use-case etc.
[Saturday 11 September 2010] [08:52:10] <pieter_hintjens> yeah
[Saturday 11 September 2010] [08:52:25] <icy> for the non blocking part I’d argue it makes it even easier given the atomicity of messages
[Saturday 11 September 2010] [08:53:50] <guido_g> we had a lot of trouble w/ these callbacks
[Saturday 11 September 2010] [08:54:32] <guido_g> one problem is that you are still in the library
[Saturday 11 September 2010] [08:54:45] <guido_g> which leads to more or less well known constraints
[Saturday 11 September 2010] [08:55:06] <pieter_hintjens> threading issues etc.
[Saturday 11 September 2010] [08:55:07] <guido_g> i wouldn’t go that route again
[Saturday 11 September 2010] [08:55:15] <pieter_hintjens> the reactor pattern looks more sane
[Saturday 11 September 2010] [08:55:17] <guido_g> pieter_hintjens: oh yes!
[Saturday 11 September 2010] [08:55:27] <guido_g> to both
[Saturday 11 September 2010] [08:55:36] <pieter_hintjens> what i’d love to make is a Libero/0MQ at some stage
[Saturday 11 September 2010] [08:56:04] <guido_g> sorry, what would that be?
[Saturday 11 September 2010] [08:56:08] <pieter_hintjens> well, something like that, anyhow: state machines where events are 0MQ messages
[Saturday 11 September 2010] [08:56:42] <pieter_hintjens> you turn the loop inside out, the loop calls your code and feeds it incoming messages as events
[Saturday 11 September 2010] [08:56:49] <pieter_hintjens> and your code goes through states
[Saturday 11 September 2010] [08:56:57] <guido_g> makes sense
[Saturday 11 September 2010] [08:57:13] <pieter_hintjens> it’s a pattern that can be terribly useful but it can be hard to grok as well
[Saturday 11 September 2010] [08:57:33] <pieter_hintjens> only worth doing for more complex 0MQ apps but I suspect we’re heading that way
[Saturday 11 September 2010] [08:57:46] <pieter_hintjens> devices that manage 3+ sockets
[Saturday 11 September 2010] [08:57:46] <guido_g> sure
[Saturday 11 September 2010] [08:59:52] <icy> yes this is what I was refering to. you give a socket a function to call upon receiving a message. this way you don’t need to care about searching which sockets got a message after a poll
[Saturday 11 September 2010] [09:00:12] <icy> zmq would do it for you and can therefor use the faster implementation possible for a given platform
[Saturday 11 September 2010] [09:00:29] <pieter_hintjens> icy: you can trivially build this on top
[Saturday 11 September 2010] [09:00:36] <pieter_hintjens> perhaps a fun thing to make
[Saturday 11 September 2010] [09:00:59] <guido_g> icy: as i said, it wasn’t nice to use in a real world app
[Saturday 11 September 2010] [09:01:08] <pieter_hintjens> i can see several problems with it, though
[Saturday 11 September 2010] [09:01:32] <guido_g> plus the problems comming from the product itself
[Saturday 11 September 2010] [09:01:33] <pieter_hintjens> passing control to a function means you can’t e.g. work with multiple sockets at once
[Saturday 11 September 2010] [09:01:45] <pieter_hintjens> so you can’t do stuff like devices that read from one, write to the other
[Saturday 11 September 2010] [09:01:58] <pieter_hintjens> or you need to create and pass around increasingly complex contexts
[Saturday 11 September 2010] [09:01:58] <guido_g> you can’t even do a proper error handling :)
[Saturday 11 September 2010] [09:02:19] <guido_g> which then slow you down even more
[Saturday 11 September 2010] [09:02:24] <pieter_hintjens> but it could be fun to experiment with
[Saturday 11 September 2010] [09:02:48] <pieter_hintjens> icy: take a look at the various reactors people like Matt Weinstein and Chuck Remes have made
[Saturday 11 September 2010] [09:04:08] <icy> of course you can do proper error handling. maybe your experience was because of a poor product, not the methodology itself guido_g
[Saturday 11 September 2010] [09:04:32] <icy> but yea to handle multiple sockets you need to pass around also some additional data
[Saturday 11 September 2010] [09:05:05] <guido_g> callbacks are generally not the first choice, i did X11 and Motif and learned to avoid them
[Saturday 11 September 2010] [09:05:16] <guido_g> then wllm, smame thing
[Saturday 11 September 2010] [09:05:21] <guido_g>
*same
[Saturday 11 September 2010] [09:05:51] <guido_g> if you want it, you should do it in the app
[Saturday 11 September 2010] [09:06:31] <guido_g> libs or frameworks can’t know enough of the use-cases to provide a proper solution for every user
[Saturday 11 September 2010] [09:06:58] <guido_g> this is why frameworks tend to become so big and bloated
[Saturday 11 September 2010] [09:09:01] <pieter_hintjens> icy: what i’ve found with 0MQ is that theory is pretty useless :-)
[Saturday 11 September 2010] [09:09:12] <pieter_hintjens> get your hands dirty with code and it becomes rapidly clear
[Saturday 11 September 2010] [09:09:24] <pieter_hintjens> happily what you learn is the same no matter what language you use
[Saturday 11 September 2010] [11:11:05] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 11 September 2010] [11:58:53] Quit keffo has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [13:08:35] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [13:14:50] Quit gebi has left this server (Read error: Operation timed out).
[Saturday 11 September 2010] [13:24:13] <keffo> um, push/pull is not pub/sub, or did i confuse things?
[Saturday 11 September 2010] [13:25:01] <keffo> ah right
[Saturday 11 September 2010] [13:37:30] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [13:58:22] Join old_sound has joined this channel (
~mrhyde@222.44.41.33).
[Saturday 11 September 2010] [13:59:01] Part old_sound has left this channel.
[Saturday 11 September 2010] [14:01:25] Quit xla has left this server (Ping timeout: 260 seconds).
[Saturday 11 September 2010] [14:11:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [14:14:38] Quit rbraley has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [15:04:18] <iFire> I mean is it feasible
*** on Sat Sep 11 16:30:13 2010
[Saturday 11 September 2010] [16:30:13] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [16:30:19] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [16:30:19] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [17:22:47] Quit rgl has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [18:04:34] <psino> is there any way to tell a socket to stop connecting to an endpoint, or stop listening on an endpoint?
[Saturday 11 September 2010] [18:19:59] <psino> and is there a way of gracefully shutting down a worker process that uses an incoming PULL socket?
[Saturday 11 September 2010] [18:37:44] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 11 September 2010] [18:53:43] Quit troutwine has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [19:06:05] Quit plq has left this server (Ping timeout: 258 seconds).
[Saturday 11 September 2010] [23:22:58] Quit rgl has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [23:33:09] Quit kooroo has left this server (Quit: HydraIRC ->
http://www.hydrairc.com <-).
[Sunday 12 September 2010] [00:22:53] Quit Samy has left this server (
*.net
*.split).
[Sunday 12 September 2010] [00:22:54] Quit mivert has left this server (
*.net
*.split).
[Sunday 12 September 2010] [01:01:42] Quit Samy has left this server (Excess Flood).
[Sunday 12 September 2010] [01:01:43] Quit rbraley has left this server (Excess Flood).
[Sunday 12 September 2010] [01:02:03] Quit soren has left this server (Remote host closed the connection).
[Sunday 12 September 2010] [01:18:59] Quit Tasser has left this server (
*.net
*.split).
[Sunday 12 September 2010] [01:42:38] Quit beppu has left this server (Ping timeout: 245 seconds).
[Sunday 12 September 2010] [01:49:53] Quit xla has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [02:12:18] Quit jsimmons has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [03:36:03] <pieter_hintjens> iFire: I’m going to make an example of that in Ch3
[Sunday 12 September 2010] [03:36:06] <pieter_hintjens> of the guide
*** on Sun Sep 12 03:46:31 2010
[Sunday 12 September 2010] [03:46:31] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [03:46:37] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [03:46:37] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [03:49:06] <iFire> pieter_hintjens chapater ch3 where
[Sunday 12 September 2010] [03:50:47] <iFire> where will the chapter 3 be pieter_hintjens?
[Sunday 12 September 2010] [03:51:20] <guido_g> user guid
[Sunday 12 September 2010] [03:51:22] <guido_g> +e
[Sunday 12 September 2010] [07:53:47] Quit psino has left this server (Ping timeout: 255 seconds).
[Sunday 12 September 2010] [08:23:28] Quit rgl has left this server (Ping timeout: 255 seconds).
[Sunday 12 September 2010] [09:12:21] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [10:14:10] Quit psino has left this server (Quit: psino).
[Sunday 12 September 2010] [10:45:27] Quit omarkj has left this server (Quit: omarkj).
[Sunday 12 September 2010] [11:44:45] Quit
benoitc_ has left this server (Changing host).
[Sunday 12 September 2010] [11:45:00] Nick
benoitc_ is now known as benoitc.
[Sunday 12 September 2010] [12:58:50] Quit jsimmons has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [13:22:40] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [13:27:37] Quit
keffo_ has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [13:47:41] <mankins> messages can be of arbitrary length, right?
[Sunday 12 September 2010] [13:56:35] <icy> they have to fit in ram
[Sunday 12 September 2010] [14:21:18] Quit gebi has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [14:31:50] Quit
gebi_ has left this server (Ping timeout: 264 seconds).
[Sunday 12 September 2010] [14:49:05] <mankins> i’m having some issue with messages getting corrupted on the wire.
[Sunday 12 September 2010] [14:49:29] <mankins> trying to isolate it.
[Sunday 12 September 2010] [14:49:54] <mankins> there are known issues with 2.0.9, are there? (I checked the bug list, and nothing jumped out)
[Sunday 12 September 2010] [14:52:35] Quit gebi has left this server (Ping timeout: 252 seconds).
[Sunday 12 September 2010] [15:11:32] Quit gebi has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [15:17:39] <mankins> hrm, maybe it was the perl serializer.
[Sunday 12 September 2010] [15:18:14] <mankins> i was using send/recv but changed to send_as/recv and it seems to be better. <shrug>
[Sunday 12 September 2010] [15:34:31] Quit gebi has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [15:44:10] Quit rgl has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [15:53:05] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [15:54:10] Quit
keffo__ has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [16:00:02] Quit gebi has left this server (Ping timeout: 264 seconds).
[Sunday 12 September 2010] [16:03:28] Quit mankins has left this server (Quit: mankins).
[Sunday 12 September 2010] [16:19:41] Quit rgl has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [16:32:54] Quit
soren_ has left this server (Changing host).
[Sunday 12 September 2010] [16:33:44] Nick
soren_ is now known as soren.
[Sunday 12 September 2010] [16:45:56] Quit plq has left this server (Quit: Leaving).
[Sunday 12 September 2010] [17:11:53] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [17:39:22] Part bobs has left this channel.
[Sunday 12 September 2010] [18:28:18] <kenkeiter> Looking for some input on a decentralized pub-sub/dequeue model with zeromq
[Sunday 12 September 2010] [18:30:23] <kenkeiter> need to be able to publish messages in a specific category, subscribe to that category and (hopefully) pull jobs from it in a load-balanced way.
[Sunday 12 September 2010] [18:30:28] <kenkeiter> Any suggestions?
[Sunday 12 September 2010] [19:26:18] Quit
keffo__ has left this server (Ping timeout: 272 seconds).
[Sunday 12 September 2010] [19:26:22] <sleeperbot> Hey guys, wondering if I could get some idea on what’s going on with my installation of zmq
[Sunday 12 September 2010] [19:26:39] <sleeperbot> been using zeromq and my CPU usage shot up x100. I’m doing Upstream/Downstream, using the node.js wrapper.
[Sunday 12 September 2010] [19:26:46] <sleeperbot> Leaving connections open on both ends to send and receive
[Sunday 12 September 2010] [19:32:15] <kenkeiter> Don’t know a tremendous amount about zmq’s internals, but my suggestion would be to verify that you have the correct number of zmq threads spawned.
[Sunday 12 September 2010] [19:32:49] <kenkeiter> If you’re respawning for each connection, they may also not be released properly..
[Sunday 12 September 2010] [19:32:57] <kenkeiter> Which could pose another problem.
*** on Sun Sep 12 19:37:10 2010
[Sunday 12 September 2010] [19:37:10] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [19:37:16] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [19:37:16] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [19:43:52] Quit andrewvc has left this server (Ping timeout: 272 seconds).
[Sunday 12 September 2010] [19:53:57] <sleeperbot> Thanks Ken, i’ll check
[Sunday 12 September 2010] [19:54:21] <sleeperbot> I’m not respawning so it could be the node.js driver
[Sunday 12 September 2010] [19:59:18] Quit GeekGod has left this server (Quit: GeekGod).
[Sunday 12 September 2010] [20:00:10] <kenkeiter> sleeperbot: it was worth a try.. did you verify that you’re only running one or two messaging threads?
[Sunday 12 September 2010] [20:00:13] <andrewvc> I’ve been curious as to how stable the node driver is
[Sunday 12 September 2010] [20:00:20] <kenkeiter> :/
[Sunday 12 September 2010] [20:03:14] <sleeperbot> I’m looking up how to do that
[Sunday 12 September 2010] [20:03:52] <sleeperbot> do you know what I can type in the command line to bring that info up?
[Sunday 12 September 2010] [20:04:03] <kenkeiter> sleeperbot: which platform?
[Sunday 12 September 2010] [20:04:25] <sleeperbot> unix
[Sunday 12 September 2010] [20:04:30] <sleeperbot> ubuntu karmic
[Sunday 12 September 2010] [20:07:43] <kenkeiter> htop might work.. haven’t done it under
*ni
[Sunday 12 September 2010] [20:14:13] <sleeperbot> I see 3 versions of my node.js stream and web servers
[Sunday 12 September 2010] [20:14:27] <sleeperbot> don’t see anything related to zmq
[Sunday 12 September 2010] [20:15:25] <sleeperbot> killed the extraneous processes, will check if anything changed in cpu usage
*** on Sun Sep 12 20:17:38 2010
[Sunday 12 September 2010] [20:17:38] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [20:17:43] Part andrewvc has left this channel.
[Sunday 12 September 2010] [20:17:44] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [20:17:44] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [20:21:22] Quit bennymack has left this server (Ping timeout: 252 seconds).
[Sunday 12 September 2010] [20:33:24] Quit psino has left this server (Quit: psino).
[Sunday 12 September 2010] [20:39:02] Quit jsimmons has left this server (Ping timeout: 264 seconds).
*** on Sun Sep 12 21:09:03 2010
[Sunday 12 September 2010] [21:09:03] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [21:09:09] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [21:09:09] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [22:57:18] Part pieter_hintjens has left this channel.
[Sunday 12 September 2010] [23:25:32] Quit kenkeiter has left this server (Quit: kenkeiter).
[Sunday 12 September 2010] [23:34:09] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [23:34:24] <andrewvc> I assume that XREQ/XREP sockets apply backpressure in the same manner as PUSH/PULL and REQ/REP yes?
[Sunday 12 September 2010] [23:59:40] Quit sleeperbot has left this server (Ping timeout: 252 seconds).
[Monday 13 September 2010] [00:15:34] Quit xla has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [00:44:16] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [00:56:55] Quit bbigras has left this server (Ping timeout: 252 seconds).
[Monday 13 September 2010] [01:30:06] Quit guido_g has left this server (Quit: Boom...).
[Monday 13 September 2010] [01:53:45] Quit
keffo__ has left this server (Read error: Connection reset by peer).
*** on Mon Sep 13 02:24:03 2010
[Monday 13 September 2010] [02:24:03] Topic The channel topic is “Welcome!”.
[Monday 13 September 2010] [02:24:09] Mode Channel modes: no messages from outside, topic protection
[Monday 13 September 2010] [02:24:09] Created This channel was created on 2010-02-05 10:44.
[Monday 13 September 2010] [02:36:59] Quit rgl has left this server (Ping timeout: 255 seconds).
[Monday 13 September 2010] [02:47:16] Quit kenkeiter has left this server (Quit: kenkeiter).
[Monday 13 September 2010] [03:18:13] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [04:37:41] Join plq has joined this channel (
~plq@78.189.30.218).
[Monday 13 September 2010] [05:08:18] Part keffo has left this channel (“Leaving”).
[Monday 13 September 2010] [07:09:02] Quit omarkj has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [07:09:03] Nick
omarkj_ is now known as omarkj.
[Monday 13 September 2010] [07:28:04] <CIA-20> zeromq2: 03Martin Lucina 07master * rbe159b6 10/ src/pipe.cpp : zmq::writer_t: Add missing test for swap - http://bit.ly/aGN4bM
[Monday 13 September 2010] [07:29:19] <icy> sustrik: hi, is there any paper on the algorithm used for the lock-free queue?
[Monday 13 September 2010] [07:30:51] <sustrik> icy: there’s a very old article here:
[Monday 13 September 2010] [07:30:59] <sustrik> lot of it doesn’t apply any more
[Monday 13 September 2010] [07:31:42] <sustrik> this is what still applies: “+ Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> - Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> FoldUnfold
[Monday 13 September 2010] [07:31:42] <sustrik> Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> Introduction
[Monday 13 September 2010] [07:31:44] <sustrik> Design
[Monday 13 September 2010] [07:31:46] <sustrik> Performance
[Monday 13 September 2010] [07:31:48] <sustrik> Configuration
[Monday 13 September 2010] [07:31:50] <sustrik> Performance with polling
[Monday 13 September 2010] [07:31:52] <sustrik> Performance without polling
[Monday 13 September 2010] [07:31:54] <sustrik> Conclusion
[Monday 13 September 2010] [07:31:56] <sustrik> HISTORICAL WHITEPAPER
[Monday 13 September 2010] [07:31:58] <sustrik> Introduction
[Monday 13 September 2010] [07:32:00] <sustrik> Y-suite is a set of components designed for ultra-efficient passing of messages between threads within a process. Y-suite is somehow similar to local sockets, however, it is much faster.
[Monday 13 September 2010] [07:32:03] <sustrik> In version 0.1 of MQ lightweight messaging kernel, the only y-suite component available is ypipe, a lock-free and wait-free implementation of a queue. In version 0.2 ypollset is added to allow thread to interchange messages with several other threads at the same time (similar to POSIX poll function). Component known as semaphore in version 0.1 is renamed to ysemaphore in version 0.2 to mark that it belongs to y-suite. Same way, spipe i
[Monday 13 September 2010] [07:32:08] <sustrik> s renamed to ysocketpair.
[Monday 13 September 2010] [07:32:10] <sustrik> Design
[Monday 13 September 2010] [07:32:14] <sustrik> The basic means of transferring message between threads is ypipe. Messages are passed through a pipe in the standard write and read manner. Once the reader has no more messages to read from the pipe, it notifies the sender using passive synchronization and goes asleep. Passive synchronization means that the other thread is not notified directly using some kind of async signal, rather it will be notified once it tries to write the next me
[Monday 13 September 2010] [07:32:19] <sustrik> ssage to the pipe. When this happens, writer becomes aware that reader is already asleep or at least going asleep at the moment. It knows that there is new message available, so it wakes the reader up using active synchronization, i.e. actively sending wake-up event to the other thread. Active synchronisation is not provided by ypipe itself, rather by other y-suite components, to be discussed bellow. Usage of ypipe is depicted on the fol
[Monday 13 September 2010] [07:32:24] <sustrik> lowing sequence diagram:”
[Monday 13 September 2010] [07:32:26] <sustrik> yuck
[Monday 13 September 2010] [07:32:28] <sustrik> sorry
[Monday 13 September 2010] [07:33:01] <sustrik> too much text, but the last paragraph is relevant
[Monday 13 September 2010] [07:33:36] <sustrik> also see the diagram that follows the text above
[Monday 13 September 2010] [07:34:23] <ekidd> Good morning! ZeroMQ is a really nice library.
[Monday 13 September 2010] [07:35:08] <ekidd> If I’m using REQ/REP messaging with multiple servers, what happens if one server is asked to handle an unusually long-running request?
[Monday 13 September 2010] [07:36:29] <ekidd> Do the clients just route requests to one of the available servers? Or do they continue to send requests to the busy server?
[Monday 13 September 2010] [07:36:30] <sustrik> ekidd: if you set high watermark, it’s queue gets eventually full and subsequent requests will be dispatches to other servers
[Monday 13 September 2010] [07:36:59] <icy> sustrik: yea I’ve read that, I guess because it is single-reader single-write, it does not suffer from the ABA problem?
[Monday 13 September 2010] [07:37:38] <ekidd> sustrik: Ah, OK. The useful high watermark in my case is very small: The servers are inherently single-threaded workers with long-running jobs. I want to keep them loaded.
[Monday 13 September 2010] [07:37:45] <sustrik> icy: what’s ABA?
[Monday 13 September 2010] [07:38:02] <ekidd> I do, however, have lots of clients and servers.
[Monday 13 September 2010] [07:39:11] <icy> sustrik: it’s one of the main problems that lock-free queues have to overcome
[Monday 13 September 2010] [07:39:23] <guido_g> it takes an unusually long time to complete the request
[Monday 13 September 2010] [07:40:01] <ekidd> icy: My clients and servers are different machines, so I don’t think the lock-free stuff is relevant. But I might be confused.
[Monday 13 September 2010] [07:40:29] <sustrik> ekidd: that’s a different conversion going on :)
[Monday 13 September 2010] [07:40:40] <ekidd> Ah, OK. I was confused. :-)
[Monday 13 September 2010] [07:41:02] <guido_g> ekidd: did you see that req/req is locked to the send/recv order?
[Monday 13 September 2010] [07:41:02] <sustrik> ekidd: there’s no such thing in 0MQ as explicit ack
[Monday 13 September 2010] [07:41:27] <sustrik> so there’s no way for it to work in lock-step fashion
[Monday 13 September 2010] [07:41:32] <ekidd> guido_g: Yeah, that works for me.
[Monday 13 September 2010] [07:42:32] <sustrik> icy: it’s basically a two step process
[Monday 13 September 2010] [07:42:32] <guido_g> same for the rep side (the server)
[Monday 13 September 2010] [07:42:41] <ekidd> I basically have a farm of Windows workers that take 0.1 to (say) 60 seconds to process a job, and idle time costs money. There’s one worker per server.
[Monday 13 September 2010] [07:42:46] <guido_g> see the user guide for lots of examples and ideas
[Monday 13 September 2010] [07:43:12] <sustrik> icy: while there are messages to be read the synchronisation is done simply by moving a pointer in the linked list in atomic manner
[Monday 13 September 2010] [07:43:32] <sustrik> icy: when there are no messages to be read, the pointer becomes NULL
[Monday 13 September 2010] [07:43:32] <ekidd> I don’t mind locking the order of responses the way req/rep does: I’m talking to expensive, single-threaded Windows libraries in any case.
[Monday 13 September 2010] [07:44:01] <icy> sustrik: understood so far and it seems it does not suffer from aba, just was curious if there was real proof of the correctness of the algorithm
[Monday 13 September 2010] [07:44:05] <sustrik> icy: reader goes asleep and standard inter-thread mechanism (socketpair) is used to wake it up
[Monday 13 September 2010] [07:44:16] <ekidd> But I want to maximize utilization of those expensive libraries.
[Monday 13 September 2010] [07:44:23] <sustrik> icy: no
[Monday 13 September 2010] [07:44:29] <sustrik> want to prove it?
[Monday 13 September 2010] [07:44:46] <icy> that would take more time than I have probably :)
[Monday 13 September 2010] [07:45:08] <ekidd> As long as zeromq clients respect the individual server’s high water marks and route requests to another worker, everything will work fine.
[Monday 13 September 2010] [07:46:09] <ekidd> I’m going to write some tests (of course). I just wanted to know whether I was even trying something sane. :-)
[Monday 13 September 2010] [07:48:01] <ekidd> Many thanks for your advice, folks!
[Monday 13 September 2010] [07:54:15] <ekidd> guido_g: Excellent. It definitely has the right semantics. I’ll still need to find out whether it does the right thing, performance-wise, with large messages and queues that are often at their high water marks.
[Monday 13 September 2010] [08:37:13] Quit rgl has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [08:39:00] Part gebi has left this channel.
[Monday 13 September 2010] [08:51:31] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [09:27:32] Quit plq has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [09:33:25] Quit guido_g has left this server (Quit: Boom...).
[Monday 13 September 2010] [09:43:13] Quit rbraley has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [09:44:53] <cremes> while writing some specs for my bindings this weekend i came across a few issues with SWAP, RECOVERY_IVL and RATE
[Monday 13 September 2010] [09:45:08] <cremes> all 3 of those take signed 64-bit integers for input
[Monday 13 September 2010] [09:45:32] <cremes> they also do not return an error when passed a negative number even though that doesn’t make any sense
[Monday 13 September 2010] [09:45:56] <cremes> should the library return an error for negative numbers or should my bindings take care of that issue?
[Monday 13 September 2010] [09:49:52] Join plq has joined this channel (
~plq@78.189.30.218).
[Monday 13 September 2010] [09:55:33] Quit jsimmons has left this server (Ping timeout: 245 seconds).
[Monday 13 September 2010] [09:58:11] <ptrb> I start the server, it sits at zmq_recv(), great; i run the client, it runs fine and exits, but the server never receives anything. ideas?
[Monday 13 September 2010] [10:00:05] <pieterh> cremes: i think all the setsockopt types need to be reviewed for 3.0
[Monday 13 September 2010] [10:00:41] <cremes> pieterh: ok; so should i open bugs for those against the 2.1.x branch?
[Monday 13 September 2010] [10:00:42] <pieterh> but certainly if they are signed get a negative value that should return EINVAL
[Monday 13 September 2010] [10:00:51] <pieterh> yup, even 2.0.x IMO
[Monday 13 September 2010] [10:00:57] <cremes> ok, i’ll do that now
[Monday 13 September 2010] [10:01:21] <pieterh> ptrb: looking at it...
[Monday 13 September 2010] [10:02:01] <ptrb> pieterh: thx; I’m guessing there’s some setup step I’ve overlooked
[Monday 13 September 2010] [10:02:23] <pieterh> ptrb: try ‘ps’
[Monday 13 September 2010] [10:02:31] <pieterh> imo you have a second copy of the server running
[Monday 13 September 2010] [10:02:52] <pieterh> (though it would assert then...)
[Monday 13 September 2010] [10:03:11] <pieterh> sorry, forget I said that plz
[Monday 13 September 2010] [10:03:11] <ptrb> hmm, no, but maybe something else is sitting on 5001, let me try changing that
[Monday 13 September 2010] [10:03:22] <pieterh> ptrb: client writes a message and then closes & exits
[Monday 13 September 2010] [10:03:28] <pieterh> two things: (a) it should wait for a reply
[Monday 13 September 2010] [10:03:37] <pieterh> (b) if it does not want to wait, it can’t exit immediately
[Monday 13 September 2010] [10:03:44] <pieterh> you need to read the users guide
[Monday 13 September 2010] [10:04:00] <ptrb> I have.
[Monday 13 September 2010] [10:04:10] <pieterh> 0mq/2.0.x loses data if you close the socket while there is data in flight
[Monday 13 September 2010] [10:04:11] <cremes> ptrb: are you starting the server first?
[Monday 13 September 2010] [10:04:16] <ptrb> Of course.
[Monday 13 September 2010] [10:04:23] <pieterh> send/close is not going to work
[Monday 13 September 2010] [10:04:28] <pieterh> send/recv/close is ok
[Monday 13 September 2010] [10:04:32] <pieterh> send/sleep/close is ok
[Monday 13 September 2010] [10:04:32] <ptrb> OK, so, do I need to recv() in the clie... k
[Monday 13 September 2010] [10:04:34] <cremes> ah yes, that’s right
[Monday 13 September 2010] [10:04:43] <cremes> do a sleep before exiting
[Monday 13 September 2010] [10:04:45] <ptrb> even if I don’t post anything back explicitly?
[Monday 13 September 2010] [10:04:47] <pieterh> prtb: either a recv
[Monday 13 September 2010] [10:04:59] Nick
bbigras_ is now known as bbigras.
[Monday 13 September 2010] [10:05:07] <pieterh> prtb if you’re using REQ and REP sockets, you should be doing send/recv and recv/send
[Monday 13 September 2010] [10:05:20] <pieterh> if you want to just send 1 message as such use PUSH/PULL
[Monday 13 September 2010] [10:05:39] <pieterh> it’s not a biggie
[Monday 13 September 2010] [10:05:52] <pieterh> the problem here is not giving the client process time to send its data
[Monday 13 September 2010] [10:05:53] <ptrb> I’m doing something vaguely RPC-ish, so I guess if I want to represent a void blah(); I still have to send something back
[Monday 13 September 2010] [10:06:06] <pieterh> or else use XREQ/XREP
[Monday 13 September 2010] [10:06:09] <ptrb> yeah it makes sense, sure. thanks. i guess it’s just not explicit anywhere in the docs (afaict)
[Monday 13 September 2010] [10:06:18] <pieterh> rtfug... :-)
[Monday 13 September 2010] [10:06:22] <pieterh> it is explicit in there
[Monday 13 September 2010] [10:06:31] <ptrb> i have; if you want to point me to the sentence in question I’m happy to be made a fool
[Monday 13 September 2010] [10:06:46] * pieterh goes off to search for ‘sleep’...
[Monday 13 September 2010] [10:07:11] <pieterh> Note that we do sleep (1); before exiting the ventilator. This is a hack that gets around MQ/2.0’s design, which discards messages that have not yet been sent, if you exit the program too soon. If you are using MQ/2.1 you can remove this sleep statement.
[Monday 13 September 2010] [10:07:43] <ptrb> eh.
[Monday 13 September 2010] [10:08:19] <pieterh> it’s the first example that has this problem, so I explain it there
[Monday 13 September 2010] [10:08:29] <pieterh> the hello world client waits for an answer
[Monday 13 September 2010] [10:08:38] <pieterh> and the pubsub example never exits
[Monday 13 September 2010] [10:09:01] <pieterh> maybe i should put it in bold...
[Monday 13 September 2010] [10:09:13] <pieterh> and repeat this, it’s a common fault
[Monday 13 September 2010] [10:11:29] <ptrb> if you’re willing to take some constructive criticism about the documentation, i’d say that while example-based docs are great, when I have a specific problem (like this) I find there isn’t really a way I can find a solution; there’s no idioms or implementation details or whatever to search through (as far as I’ve found)
[Monday 13 September 2010] [10:13:56] <ptrb> but!! but but, thank you :)
[Monday 13 September 2010] [10:19:21] <ptrb> hmm, recv on the client side hangs... is there not some zmq_flush or something I can call?
[Monday 13 September 2010] [10:20:33] <guido_g> no
[Monday 13 September 2010] [10:20:40] <ptrb> poop :|
[Monday 13 September 2010] [10:21:06] <guido_g> pardon?
[Monday 13 September 2010] [10:21:19] <ptrb> that was an expression of mild disappointment
[Monday 13 September 2010] [10:31:20] Quit plq has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [10:40:40] Quit mato has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [11:04:45] Quit rbraley has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [11:16:08] Quit mivert has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [11:16:51] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [11:35:55] <pieterh> ptrb: still there?
[Monday 13 September 2010] [11:36:04] <pieterh> sorry, was in a meeting
[Monday 13 September 2010] [11:36:19] <ptrb> yeah sure
[Monday 13 September 2010] [11:36:54] <pieterh> making a problem driven section in the guide would be good
[Monday 13 September 2010] [11:37:01] <pieterh> did you find out why your client hangs?
[Monday 13 September 2010] [11:37:29] <ptrb> No, I just threw a sleep in there and moved on to bigger, even more problematic things :)
[Monday 13 September 2010] [11:37:41] <ptrb> a problem-driven section would be good, but it’ll never be comprehensive
[Monday 13 September 2010] [11:38:09] <ptrb> FWIW I think a good documentation model would be ZeroC’s ICE, which has a really comprehensive .pdf
[Monday 13 September 2010] [11:38:16] <pieterh> “did not get a message” is a pretty classic stumbling block
[Monday 13 September 2010] [11:38:23] <ptrb> yeah, fair
[Monday 13 September 2010] [11:38:33] <pieterh> i’ll write a flowchart
[Monday 13 September 2010] [11:40:41] <ptrb> now, i’m working on an implementation based on the multithreaded code in the user guide, and i’m getting infinite size-0 messages on the server side after sending one legitimate message from a client
[Monday 13 September 2010] [11:40:48] <ptrb> ever hear of something like this?
[Monday 13 September 2010] [11:42:33] <ptrb> sorry, based on the multithreaded server in the introduction doc
[Monday 13 September 2010] [11:43:41] <cremes> ptrb: i’ve never seen that... you say your code is “based on” the example; it’s always a good idea to start from code that you know works and modify from there
[Monday 13 September 2010] [11:43:49] <cremes> sounds like your mods broke it
[Monday 13 September 2010] [11:44:08] <cremes> the easiest way to find the failure is to revert back to the original “good” code and slowly modify it to your specifications
[Monday 13 September 2010] [11:44:27] <ptrb> yeah. i know. i’m trying to drop the server into an existing process to provide a zmq “layer”, so there’s not really any way to iterate my way to where I am now.
[Monday 13 September 2010] [11:45:07] <ptrb> i guess i can try taking out some functionality.
[Monday 13 September 2010] [11:45:20] <cremes> did you change the code that sends 0mq messages?
[Monday 13 September 2010] [11:48:35] <ptrb> yes; in ways i initially thought were inconsequential, but i suppose i’m in an assumption-revalidating mood :)
[Monday 13 September 2010] [11:49:15] <ptrb> as a meta-comment, it’s really great you guys are hanging out on irc to help folks; zmq is a great project and this is a great resource.
[Monday 13 September 2010] [11:50:54] <ptrb> aha! so, if i zmq_recv(), get a message, and don’t zmq_send() something in the server, subsequent zmq_recv()s have the effect of not blocking
[Monday 13 September 2010] [11:51:09] <ptrb> ...which seems quite strange to me
[Monday 13 September 2010] [11:51:57] <cremes> ptrb: what kind of socket are you using on this server side?
[Monday 13 September 2010] [11:53:06] <cremes> because that behavior doesn’t sound right; the zmq_recv() call is returning an error, right?
[Monday 13 September 2010] [11:54:04] <ptrb> yeah, it returns -1 EAGAIN
[Monday 13 September 2010] [11:54:11] <ptrb> i believe it’s EAGAIN, at least.
[Monday 13 September 2010] [11:56:31] <ptrb> the topology is the multithread server example in the intro doc: public tcp XREP endpoint, managed by one thread running zmq_device(ZMQ_QUEUE, ...), forwarding via XREQ to an inproc endpoint, being consumed by worker threads binding to REP
[Monday 13 September 2010] [12:02:56] <cremes> can you provide a code pastie?
[Monday 13 September 2010] [12:03:14] <ptrb> it won’t be complete, but sure, one sec...
[Monday 13 September 2010] [12:03:38] <cremes> it doesn’t need to be complete... i want to see the code that sets up the socket and calls recv on it
[Monday 13 September 2010] [12:04:03] <ptrb> the worker thread ultimately responsible for processing the recv, right?
[Monday 13 September 2010] [12:04:39] <cremes> whatever code is returning -1 EAGAIN
[Monday 13 September 2010] [12:06:16] Quit xla has left this server (Quit: leaving).
[Monday 13 September 2010] [12:08:10] <cremes> ptrb: in your DEBUG statement, also print out the value of zmq_strerror()
[Monday 13 September 2010] [12:08:27] <cremes> i need more information to figure this out
[Monday 13 September 2010] [12:09:15] Quit xla has left this server (Client Quit).
[Monday 13 September 2010] [12:09:18] <ptrb> Operation cannot be accomplished in current state
[Monday 13 September 2010] [12:10:02] <cremes> ah, then there we have it; with a REP socket you can’t call recv again until you have subsequently called send
[Monday 13 September 2010] [12:10:16] <cremes> it needs that recv/send/recv pattern because it maintains a small internal state machine
[Monday 13 September 2010] [12:10:23] <ptrb> oh, interesting
[Monday 13 September 2010] [12:10:24] <cremes> that’s the whole point of the REQ/REP socket pattern
[Monday 13 September 2010] [12:10:24] <ptrb> ok
[Monday 13 September 2010] [12:10:34] <ptrb> see, this is useful! this should be on a website somewhere :)
[Monday 13 September 2010] [12:10:40] <cremes> the worker is supposed to respond when it is done, right?
[Monday 13 September 2010] [12:10:50] <ptrb> well, my thought is that it may optionally respond
[Monday 13 September 2010] [12:10:51] <cremes> it is for sure
[Monday 13 September 2010] [12:10:58] <ptrb> but if it has to respond, that’s fine too
[Monday 13 September 2010] [12:11:07] <cremes> if you want it to be optional, use XREP sockets
[Monday 13 September 2010] [12:11:25] <cremes> that kind of socket does not enforce the recv/send/recv pattern
[Monday 13 September 2010] [12:11:45] <ptrb> I suspect there is more to XREP than simply dropping that enforcement, though
[Monday 13 September 2010] [12:12:53] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [12:13:51] <cremes> ptrb: not really; REP sockets are built on top of the XREP socket
[Monday 13 September 2010] [12:14:00] <ptrb> hmm, interesting
[Monday 13 September 2010] [12:14:14] <cremes> REP sockets know how to “route” their responses back over multiple hops
[Monday 13 September 2010] [12:14:29] <cremes> you need to do a little extra work when using an XREP socket to retain that functionality
[Monday 13 September 2010] [12:18:53] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [12:26:08] <ptrb> curiouser and curiouser
[Monday 13 September 2010] [12:26:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [12:39:20] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Monday 13 September 2010] [13:10:03] Quit bennymack has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [13:13:00] Quit ekidd has left this server (Quit: ekidd).
[Monday 13 September 2010] [13:19:56] Quit omarkj has left this server (Quit: omarkj).
[Monday 13 September 2010] [13:27:35] Join AlexB has joined this channel (
alex@brasetvik.com).
[Monday 13 September 2010] [14:12:09] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [14:13:16] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [14:45:08] Quit ReinH has left this server (Ping timeout: 245 seconds).
[Monday 13 September 2010] [15:59:18] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [16:08:23] Quit rphillips has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [16:37:48] <ModusPwnens> Hi guys, I have a question on req/rep topology
[Monday 13 September 2010] [16:38:17] <ModusPwnens> Previously, I have been doing benchmarking with the subscriber/publisher toplogy, but I wanted to see what results I would get with req/rep
[Monday 13 September 2010] [16:38:52] <ModusPwnens> and I was wondering if there is anything else I need to do besides the obvious change of the socket types and adding in addition send/recv function calls to avoid blocking the code
[Monday 13 September 2010] [16:39:26] <ModusPwnens> because i noticed after I did those things, rather than sending a message of X bytes, it sends X messages of 1 byte
[Monday 13 September 2010] [16:41:05] <ModusPwnens> Actually, I lied. It seems to just send a lot of zero byte messages
[Monday 13 September 2010] [16:41:49] <cremes> ModusPwnens: pastie some code, because it should “just work”
[Monday 13 September 2010] [16:44:17] Quit gavinstark has left this server (Remote host closed the connection).
[Monday 13 September 2010] [16:51:29] <ModusPwnens> Actually, i figured out what it was. However, should rep/req have better or worse performance than sub/pub?
[Monday 13 September 2010] [17:07:24] <cremes> ModusPwnens: same perf but round-trip latency is higher (no such notion as round-trip latency with pub/sub)
[Monday 13 September 2010] [17:27:18] Quit plq has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [18:07:29] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [18:33:14] Quit ekidd has left this server (Quit: ekidd).
[Monday 13 September 2010] [18:46:37] Quit psino has left this server (Quit: psino).
[Monday 13 September 2010] [19:10:47] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [19:57:38] Quit rgl has left this server (Ping timeout: 264 seconds).
[Monday 13 September 2010] [20:24:01] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [20:39:05] Quit rphillips has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [20:50:48] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [20:54:32] Quit jhawk28 has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [20:59:33] Quit
jhawk28_ has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [21:05:12] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [21:10:53] Quit plq has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [21:15:35] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [21:42:44] Quit GeekGod has left this server (Quit: GeekGod).
[Monday 13 September 2010] [21:58:16] Quit grahamalot has left this server (Quit: grahamalot).
[Monday 13 September 2010] [22:00:53] Quit xla has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [22:17:53] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [22:30:44] Quit bgranger has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [22:45:29] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [23:39:45] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [00:52:17] Quit keffo has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [01:06:56] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [01:14:25] Quit psino has left this server (Quit: psino).
[Tuesday 14 September 2010] [01:55:59] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 14 September 2010] [01:56:36] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 14 September 2010] [01:58:14] Quit rgl has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [02:57:32] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [03:14:11] Quit jond has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [04:45:14] <pieterh> sustrik: ping?
[Tuesday 14 September 2010] [04:45:31] <sustrik> pieterh: pong
[Tuesday 14 September 2010] [04:45:42] <pieterh> is there any zero copy mechanism on reading messages?
[Tuesday 14 September 2010] [04:45:51] <pieterh> i.e. directly into application buffers
[Tuesday 14 September 2010] [04:46:45] <sustrik> no, you get the buffer from 0mq
[Tuesday 14 September 2010] [04:46:53] <sustrik> you can then store it as long as you wish
[Tuesday 14 September 2010] [04:47:12] <pieterh> ok, thx
[Tuesday 14 September 2010] [05:04:05] <ptrb> so you can
`zero-copy’ the buffer data beyond the lifetime of the zmq_msg_t?
[Tuesday 14 September 2010] [05:08:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [05:10:23] <sustrik> ptrb: no, but you can keep zmq_msg_t around as long as you wish
[Tuesday 14 September 2010] [05:10:34] <ptrb> roger.
[Tuesday 14 September 2010] [05:18:18] <pieterh> ptrb: I’ve added a problem solver to the guide
[Tuesday 14 September 2010] [05:20:46] <ptrb> cool
[Tuesday 14 September 2010] [05:21:00] <ptrb> the “Yes” path on “Are you using XREP sockets?” isn’t labeled :D
[Tuesday 14 September 2010] [05:21:08] <pieterh> thanks...!
[Tuesday 14 September 2010] [05:21:22] <ptrb> but that is definitely helpful
[Tuesday 14 September 2010] [05:21:40] <ptrb> i’m a little overwhelmed by the sheer number of different “guides” and “manuals” and whatnot on the docs page(s), though
[Tuesday 14 September 2010] [05:23:39] <pieterh> ptrb: yeah, standard process, lots of growth and then consolidation
[Tuesday 14 September 2010] [05:23:50] <pieterh> stick with www.zeromq.com for starters
[Tuesday 14 September 2010] [05:24:44] <pieterh> really there should only be two books, the guide and the reference manual
[Tuesday 14 September 2010] [05:24:55] <ptrb> sure
[Tuesday 14 September 2010] [05:25:16] <ptrb> there’s a solid case for a quick-start (~1h) tutorial as well, I think; maybe that’s like ch.1 of the guide
[Tuesday 14 September 2010] [05:25:39] <pieterh> yup
[Tuesday 14 September 2010] [05:25:55] <pieterh> plus lots of tutorials and articles from the rest of the community
[Tuesday 14 September 2010] [05:41:04] <keffo> pieterh, I have a concurrency issue that I cant really figure out!
[Tuesday 14 September 2010] [05:41:14] <pieterh> keffo: shoot
[Tuesday 14 September 2010] [05:41:31] <pieterh> have you checked the new problem solver?
[Tuesday 14 September 2010] [05:41:40] <keffo> lemme check
[Tuesday 14 September 2010] [05:42:57] <keffo> pieterh, Think client, double lru-queue(for outoing tasks, and for incoming results), then an additional ‘regular’ queue on each machine, which provides N worker processes..
[Tuesday 14 September 2010] [05:43:14] <keffo> pieterh, Theyre all sep. processes, no threading involved anywhere apart from zmq internals..
[Tuesday 14 September 2010] [05:43:22] <pieterh> ok
[Tuesday 14 September 2010] [05:43:53] <pieterh> what is the symptom you are seeing?
[Tuesday 14 September 2010] [05:44:29] <keffo> One worker, all is fine, I’ve done 100000 tasks back’n’forth with no problem, so the handling of data is sound.. Now if I do more than one worker processes, -sometimes- I get jumbled data, so it’s clearly a concurrency issue, but I cant really tell how or where
[Tuesday 14 September 2010] [05:45:12] <pieterh> jumbled data = one message has corrupted data or messages are not in order?
[Tuesday 14 September 2010] [05:45:34] <keffo> I guess both? =)
[Tuesday 14 September 2010] [05:45:43] <keffo> lemme trigger it and check the logs
[Tuesday 14 September 2010] [05:45:48] <pieterh> ok
[Tuesday 14 September 2010] [05:46:48] <keffo> there we go
[Tuesday 14 September 2010] [05:47:53] <keffo> so this scenario has multiple worker processes, but I send out batches or bursts of tasks of different granularity.. With 1 it always works, that is, send one task, wait for that task to complete.. The problem pops up when doing more than one, say 10
[Tuesday 14 September 2010] [05:48:00] <keffo> which is the case now
[Tuesday 14 September 2010] [05:48:31] <pieterh> keffo: you still need to explain what the “problem” actually is
[Tuesday 14 September 2010] [05:48:57] <pieterh> what is the difference between working and non-working in terms of what you see in the logs?
[Tuesday 14 September 2010] [05:49:43] <keffo> well, sometimes routing info is jumbled.. I expect uuid + nil + payload, but sometimes that uuid can get very weird
[Tuesday 14 September 2010] [05:50:01] <keffo> and likewise the payload itself can get messed up
[Tuesday 14 September 2010] [05:50:12] <pieterh> ok, so your message envelopes are wrong
[Tuesday 14 September 2010] [05:50:25] <keffo> They would be wrong with just one message too :)
[Tuesday 14 September 2010] [05:50:32] <keffo> or one worker proc.
[Tuesday 14 September 2010] [05:50:43] <pieterh> most probably an error in your code that creates/copies envelopers
[Tuesday 14 September 2010] [05:50:47] <pieterh>
*envelopes
[Tuesday 14 September 2010] [05:51:19] <pieterh> the LRU queue stuff is pretty delicate
[Tuesday 14 September 2010] [05:53:47] <keffo> I dont see the relevance to be honest, I can process 1M tasks properly if I wait for each message(or if I have only one worker processing them).. If there was an inherent problem in the envelope construction/usage, surely that would pop up there as well, but it has -never- happened in that scenario..
[Tuesday 14 September 2010] [05:54:02] Quit jsimmons has left this server (Ping timeout: 252 seconds).
[Tuesday 14 September 2010] [05:54:40] <pieterh> keffo: when problem solving you don’t care when it works
[Tuesday 14 September 2010] [05:54:52] <pieterh> if you can reproduce a problem, that’s the interesting thing
[Tuesday 14 September 2010] [05:55:06] <pieterh> things often work by accident...
[Tuesday 14 September 2010] [05:55:21] <pieterh> so your workers are REQ and you’re talking to them with an XREP?
[Tuesday 14 September 2010] [05:55:54] <keffo> yeah
[Tuesday 14 September 2010] [05:56:12] <pieterh> do you store the worker identity anywhere?
[Tuesday 14 September 2010] [05:56:23] <keffo> I can reproduce it, but with irregularities one has come to expect from multithreaded :)
[Tuesday 14 September 2010] [05:56:31] <keffo> yeah, all connections are known
[Tuesday 14 September 2010] [05:57:24] <pieterh> so the obvious thing to look at would be how you reconstruct the envelopes and where the worker addresses come from
[Tuesday 14 September 2010] [05:57:32] <pieterh> and that you’re not using the wrong data
[Tuesday 14 September 2010] [05:57:44] Quit omarkj has left this server (Read error: Connection reset by peer).
[Tuesday 14 September 2010] [05:58:34] <pieterh> you presumably have logic that is vulnerable to multiple workers sending back stuff at the same time
[Tuesday 14 September 2010] [05:59:04] <pieterh> maybe you’re not using SNDMORE properly in the worker when sending
[Tuesday 14 September 2010] [05:59:22] <pieterh> so messages arrive mixed together rather than atomically
[Tuesday 14 September 2010] [05:59:31] * pieterh has no idea if that is even possible...
[Tuesday 14 September 2010] [05:59:53] <keffo> right, I dont see that being possible either, as it properly routes across 3-4 queues?
[Tuesday 14 September 2010] [06:00:08] <pieterh> you have assertions on everything?
[Tuesday 14 September 2010] [06:00:20] <keffo> think so, yeah
[Tuesday 14 September 2010] [06:00:34] <pieterh> so every single frame is checked? e.g. RCVMORE where expected, empty frame where expected, etc.
[Tuesday 14 September 2010] [06:00:50] <keffo> yes
[Tuesday 14 September 2010] [06:01:05] <pieterh> the only think I can think of right now is that you can send what looks like a valid envelope but it won’t survive concurrency
[Tuesday 14 September 2010] [06:02:21] <keffo> What would cause multiple incoming multipart-messages to break symmetry with more than one sender, in a singlethreaded process?
[Tuesday 14 September 2010] [06:02:34] <keffo> that’s what I dont get :)
[Tuesday 14 September 2010] [06:02:47] <pieterh> well, if they are not actually multipart messages
[Tuesday 14 September 2010] [06:02:56] <pieterh> lets say you send the envelope (address, null, data)
[Tuesday 14 September 2010] [06:03:06] <pieterh> but you by mistake use normal sends on each part
[Tuesday 14 September 2010] [06:03:19] <keffo> yup
[Tuesday 14 September 2010] [06:03:21] <pieterh> it’ll be 3 messages, not 1 3-part message
[Tuesday 14 September 2010] [06:03:31] <pieterh> if you read that, it might look valid and work for routing
[Tuesday 14 September 2010] [06:03:40] <pieterh> but as soon as you have concurrency it’ll stop working
[Tuesday 14 September 2010] [06:03:56] <pieterh> like i say, i’ve never tried that, but will make a test now, after coffee
[Tuesday 14 September 2010] [06:04:12] <pieterh> i’d expect 0MQ to reject such messages...
[Tuesday 14 September 2010] [06:04:29] <keffo> hmm.. I’ll go over the code again! :)
[Tuesday 14 September 2010] [06:04:32] <pieterh> but from REQ to XREP it can’t IMO
[Tuesday 14 September 2010] [06:06:40] <keffo> cant reject?
[Tuesday 14 September 2010] [06:07:04] <sustrik> keffo: can you check whether the garbled identity you see is something that may actually your application message or whether it’s just real junk
[Tuesday 14 September 2010] [06:07:05] <sustrik> ?
[Tuesday 14 September 2010] [06:09:20] <keffo> It’s not real junk.. Sometimes for example I get route uuid that appears to be a mix of multiple uuid’s
[Tuesday 14 September 2010] [06:09:41] <keffo> clearly something breaks symmetry somehow
[Tuesday 14 September 2010] [06:09:50] <keffo> I’ll lok into the route logic again though
[Tuesday 14 September 2010] [06:11:56] <sustrik> “mix of multiple uuid’s” = junk IMO
[Tuesday 14 September 2010] [06:12:08] <sustrik> ok, a sanity check:
[Tuesday 14 September 2010] [06:12:20] <sustrik> do you use each socket exclusively from a single thread?
[Tuesday 14 September 2010] [06:12:44] <keffo> yeah
[Tuesday 14 September 2010] [06:12:54] <sustrik> what language are you using?
[Tuesday 14 September 2010] [06:13:02] <keffo> c++ and lua
[Tuesday 14 September 2010] [06:13:10] <keffo> but homebrew bindings
[Tuesday 14 September 2010] [06:13:42] <sustrik> any chance to test a pure c++ setup?
[Tuesday 14 September 2010] [06:13:55] <keffo> hehe. noo :)
[Tuesday 14 September 2010] [06:14:35] <sustrik> hm, lua binding is a project i know nothing about
[Tuesday 14 September 2010] [06:15:02] <sustrik> the bug can be there, as no such problem was yet reported for any other language
[Tuesday 14 September 2010] [06:15:30] <sustrik> ok, any chance of using anything other than lua?
[Tuesday 14 September 2010] [06:19:11] <keffo> Would take forever to rewrite
[Tuesday 14 September 2010] [06:19:20] <sustrik> ok
[Tuesday 14 September 2010] [06:19:32] <sustrik> how does the setup look like?
[Tuesday 14 September 2010] [06:19:40] <sustrik> which part is c++ and which is lua?
[Tuesday 14 September 2010] [06:20:03] <keffo> all zmq stuff is c, rest is lua.. Ie, connect/bind/send/recv(and their more-counterparts)
[Tuesday 14 September 2010] [06:20:51] <sustrik> you are mixing C code and lua code to use the same 0MQ socket?
[Tuesday 14 September 2010] [06:21:01] * sustrik doesn’t follow
[Tuesday 14 September 2010] [06:21:22] <keffo> Um no, I created bindings for sockets, so it’s the same thing
[Tuesday 14 September 2010] [06:21:50] <sustrik> hm, are you saying that 0MQ is written in C while your apps are in lua?
[Tuesday 14 September 2010] [06:22:29] <keffo> I’ve extended an application with lua yeah..
[Tuesday 14 September 2010] [06:22:34] <keffo> lua or not is besides the point really
[Tuesday 14 September 2010] [06:22:47] <keffo> All zmq related code is in C++
[Tuesday 14 September 2010] [06:23:24] <sustrik> aha, so you are _not_ using the lua binding
[Tuesday 14 September 2010] [06:23:26] <sustrik> right?
[Tuesday 14 September 2010] [06:23:44] <keffo> I am, but not the ones from the site :)
[Tuesday 14 September 2010] [06:23:50] <sustrik> your own
[Tuesday 14 September 2010] [06:23:55] <keffo> yup
[Tuesday 14 September 2010] [06:24:14] <keffo> just an abstraction layer that’s easier to work with on the lua side
[Tuesday 14 September 2010] [06:24:20] <sustrik> ack
[Tuesday 14 September 2010] [06:24:29] <sustrik> what’s the lua’s threading model?
[Tuesday 14 September 2010] [06:24:40] <keffo> inherently single threaded, but it has coroutines
[Tuesday 14 September 2010] [06:25:38] <sustrik> ok, so there’s only one OS thread for each lua process, right?
[Tuesday 14 September 2010] [06:25:42] <keffo> yes
[Tuesday 14 September 2010] [06:26:22] <sustrik> no such think as garbage collection thread or similar?
[Tuesday 14 September 2010] [06:26:25] <sustrik> thing*
[Tuesday 14 September 2010] [06:26:26] <keffo> client, loadbalancer, computenode and workers are all single processes (a computenode is simply a c++ queue dev. ‘owning’ the worker processes)
[Tuesday 14 September 2010] [06:26:52] <keffo> No, not in another thread no.. Lua does incremental gc but in the same thread
[Tuesday 14 September 2010] [06:27:11] <sustrik> ok
[Tuesday 14 September 2010] [06:27:37] <sustrik> then it looks like the problem has to be in 0MQ itself
[Tuesday 14 September 2010] [06:27:46] <sustrik> it should be reproducible in C or C++
[Tuesday 14 September 2010] [06:28:15] <sustrik> can you write a simpe C/C++ test that simulates what you are doing in your app?
[Tuesday 14 September 2010] [06:28:35] <sustrik> messaging-wise i mean, not the business logic
[Tuesday 14 September 2010] [06:28:47] <keffo> hmm.. that would take a while I think..
[Tuesday 14 September 2010] [06:29:07] <keffo> I do think I’m to blame, not zmq, I just need a better way to debug it probably :)
[Tuesday 14 September 2010] [06:30:01] <keffo> It behaves as threaded concurrency issue, but being entirely single process based..
[Tuesday 14 September 2010] [06:30:37] <sustrik> there’s an I/O thread inside 0MQ so it can be a race
[Tuesday 14 September 2010] [06:31:00] <sustrik> however, i can’t help unless i am able to reproduce it
[Tuesday 14 September 2010] [06:31:20] <keffo> Well I’d be half way safe if I could reproduce it too, but it’s quite rare :)
[Tuesday 14 September 2010] [06:31:37] <sustrik> of course, it’s still possible that you are overwriting memory in your app or something like that
[Tuesday 14 September 2010] [06:31:54] <sustrik> have you tried to valgring your binding?
[Tuesday 14 September 2010] [06:31:58] <sustrik> valgrind
[Tuesday 14 September 2010] [06:33:17] <keffo> no valgrind in win as far as I know.. Boundschecker perhaps, but I doubt that’s the problem too
[Tuesday 14 September 2010] [06:33:41] <keffo> Again, in a single-worker scenario, everything is flawless regardless of amount and load etc
[Tuesday 14 September 2010] [06:34:00] <keffo> (and none of the processes every leak anything, stays solid around 2mb mem)
[Tuesday 14 September 2010] [06:34:39] <sustrik> then it looks like a problem in the central dispatcher node
[Tuesday 14 September 2010] [06:34:50] <keffo> yeah
[Tuesday 14 September 2010] [06:34:57] <keffo> that’s the most complicated bit too
[Tuesday 14 September 2010] [06:35:09] <keffo> double lru queue device basically
[Tuesday 14 September 2010] [06:36:01] <sustrik> well, unless you are able to provide something to reproduce the problem, you are on your own here
[Tuesday 14 September 2010] [06:36:08] <keffo> I know :)
[Tuesday 14 September 2010] [06:37:02] <keffo> more logging I guess. It would be nice to have all participants publish all the data they process, and collect that in a sub app, and then be able to “play back” everything
[Tuesday 14 September 2010] [06:37:30] <ptrb> that could easily become a project as complex as zmq itself
[Tuesday 14 September 2010] [06:37:43] <ptrb> entire companies are built on the concept of capturing/replaying network traffic
[Tuesday 14 September 2010] [06:37:52] <pieterh> keffo: if you can send me your code I’ll see if I can spot anything
[Tuesday 14 September 2010] [06:38:03] <pieterh> i need the parts that read and write worker envelopes, to start with
[Tuesday 14 September 2010] [06:38:12] <keffo> sure hold on
[Tuesday 14 September 2010] [06:38:17] <keffo> wheres the pastebin?
[Tuesday 14 September 2010] [06:38:29] <pieterh> gist.github.com works well
[Tuesday 14 September 2010] [06:39:04] <keffo> But again, if the envelope code is flawed, it would be equally flawed for one worker?
[Tuesday 14 September 2010] [06:40:27] <pieterh> keffo: i explained how it could be flawed but still work for one worker
[Tuesday 14 September 2010] [06:42:22] <pieterh> keffo: i need the actual detail of sendpath, I think
[Tuesday 14 September 2010] [06:42:27] <pieterh>
*sendmore, sorry
[Tuesday 14 September 2010] [06:42:45] <keffo> That is in C, hold on
[Tuesday 14 September 2010] [06:44:52] <keffo> send() is obviously quite similar except for the flag :)
[Tuesday 14 September 2010] [06:45:56] <pieterh> keffo: it looks fine
[Tuesday 14 September 2010] [06:46:15] <pieterh> you’re sure that pairs() gives you stuff in the right order?
[Tuesday 14 September 2010] [06:46:38] <pieterh> so you have only one choice left, sadly
[Tuesday 14 September 2010] [06:46:55] <keffo> rewrite, and hope it goes away? =)
[Tuesday 14 September 2010] [06:47:05] <keffo> naa, there’s some stupid silly thing somewhere
[Tuesday 14 September 2010] [06:47:13] <pieterh> well, everything goes through these functions
[Tuesday 14 September 2010] [06:47:20] <pieterh> so put in printf statements or whatever
[Tuesday 14 September 2010] [06:47:27] <pieterh> and see when it actually goes wrong
[Tuesday 14 September 2010] [06:47:46] <keffo> Yeah, need to find a pattern..
[Tuesday 14 September 2010] [06:47:50] <pieterh> pieter’s rules #5 and #23 of debugging
[Tuesday 14 September 2010] [06:48:03] <pieterh> (a) it’s never actually a race condition
[Tuesday 14 September 2010] [06:48:14] <pieterh> (b) you can find any bug using printfs
[Tuesday 14 September 2010] [06:48:23] <keffo> except the ones in printf!
[Tuesday 14 September 2010] [06:48:37] <pieterh> that’s why C offers puts as well!
[Tuesday 14 September 2010] [06:49:39] <keffo> One thing I could do is rewrite the loadbalancer stuff in pure C
[Tuesday 14 September 2010] [06:50:11] <pieterh> keffo: there’s no point rewriting unless you have found the cause of the problem
[Tuesday 14 September 2010] [06:50:31] <ptrb> why are your rule #5 and #23 labelled (a) and (b)??
[Tuesday 14 September 2010] [06:50:31] <keffo> indeed
[Tuesday 14 September 2010] [06:51:04] <pieterh> ptrb: lol
[Tuesday 14 September 2010] [06:51:04] <keffo> pieterh, That code should be rewritten though, for performance, but I agree :)
[Tuesday 14 September 2010] [07:33:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [07:37:02] <BooTheHamster> What mean error “operation cannot be accomplished in current state” in zmq_send() call?
[Tuesday 14 September 2010] [07:38:51] <ptrb> hey i got that yesterday! :D
[Tuesday 14 September 2010] [07:39:00] <ptrb> basically means you’re violating some constraint of the socket
[Tuesday 14 September 2010] [07:39:08] <ptrb> is it ZMQ_REQ or _REP?
[Tuesday 14 September 2010] [07:39:15] <BooTheHamster> ZMQ_REQ
[Tuesday 14 September 2010] [07:39:37] <ptrb> you must call zmq_send, then zmq_recv, before you can call zmq_send again
[Tuesday 14 September 2010] [07:39:39] <BooTheHamster> I’t a client app but run withiut server
[Tuesday 14 September 2010] [07:39:39] <ptrb> are you doing that?
[Tuesday 14 September 2010] [07:39:46] <BooTheHamster> yes
[Tuesday 14 September 2010] [07:39:55] <BooTheHamster> first I call zmq_send
[Tuesday 14 September 2010] [07:40:05] <BooTheHamster> and then zmq_recv
[Tuesday 14 September 2010] [07:40:11] <ptrb> on the same socket
[Tuesday 14 September 2010] [07:40:19] <BooTheHamster> m_MsgSock->recv(&msg_rcv, ZMQ_NOBLOCK)
[Tuesday 14 September 2010] [07:40:21] <BooTheHamster> yes
[Tuesday 14 September 2010] [07:40:33] <ptrb> noblock means you may not actually have recv’d anything
[Tuesday 14 September 2010] [07:40:46] <ptrb> you have to actually recv something before you can send again (I believe–maybe someone can confirm)
[Tuesday 14 September 2010] [07:41:21] <BooTheHamster> but i want non blocking recv
[Tuesday 14 September 2010] [07:41:36] <ptrb> that’s fine, you just can’t send again until your non-blocking recv actually gets something
[Tuesday 14 September 2010] [07:41:57] <ptrb> if you want to do multiple sends without getting data back, use XREQ
[Tuesday 14 September 2010] [07:43:41] <BooTheHamster> Thank you ptrb, XREQ helps.
[Tuesday 14 September 2010] [07:44:29] <BooTheHamster> A little one question ... on server side i must use XREP or REP socket?
[Tuesday 14 September 2010] [07:45:36] <ptrb> just depends on what behavior you can tolerate there... both will work, but REP requires you to recv, send, recv, send actual data
[Tuesday 14 September 2010] [07:46:08] <ptrb>
`man zmq_socket’ for more details
[Tuesday 14 September 2010] [07:50:49] <BooTheHamster> thanks
[Tuesday 14 September 2010] [07:52:34] Part BooTheHamster has left this channel.
[Tuesday 14 September 2010] [07:52:40] <ptrb> i helped!! :D
[Tuesday 14 September 2010] [07:52:44] <ptrb> maybe
[Tuesday 14 September 2010] [07:55:42] Quit psino has left this server (Quit: psino).
[Tuesday 14 September 2010] [08:04:35] Part icy has left this channel.
[Tuesday 14 September 2010] [08:25:25] <cremes> ptrb: you’re an expert now! ;)
[Tuesday 14 September 2010] [08:46:18] Quit jsimmons has left this server (Quit: Leaving).
[Tuesday 14 September 2010] [08:48:39] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Tuesday 14 September 2010] [08:51:23] Quit xla has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [09:02:00] <pieterh> cremes: I moved your xrep-xrep recipe page into the tutorials section
[Tuesday 14 September 2010] [09:07:28] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 14 September 2010] [09:27:17] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [09:47:35] <Tasser> why use rbzmq <=> ffi-rzmq?
[Tuesday 14 September 2010] [09:54:46] <cremes> Tasser: they have slightly different APIs; the ffi-rzmq more closely tracks the C api whereas rbzmq uses strings for everything
[Tuesday 14 September 2010] [09:55:02] <cremes> also, ffi-rzmq is more compatible with ruby runtimes (jruby, rubinius, mri)
[Tuesday 14 September 2010] [09:55:08] <Tasser> strings sounds like overuse of core classes
[Tuesday 14 September 2010] [09:55:37] <cremes> well, most people will probably convert their data to string format for send/recv under ruby because it’s convenient
[Tuesday 14 September 2010] [09:55:54] <Tasser> that’s what JSON is for :-)
[Tuesday 14 September 2010] [09:56:10] <cremes> but since ffi-rzmq exposes a zmq_msg_t-like structure you can pass around C structs without doing a lot of extra copying
[Tuesday 14 September 2010] [09:56:15] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [09:56:15] <Tasser> but I suspect you can’t hardwire zmq against json
[Tuesday 14 September 2010] [09:56:18] <cremes> that might be more efficient
[Tuesday 14 September 2010] [09:56:40] <cremes> what do you mean by hardware? as far as i know, 0mq (and zmq) don’t care what you pass on the wire
[Tuesday 14 September 2010] [09:56:47] <cremes> s/hardware/hardwire/
[Tuesday 14 September 2010] [09:57:07] <Tasser> well, for filtering in subscribe
[Tuesday 14 September 2010] [09:57:28] <cremes> ah, subscribe filters at the byte-level
[Tuesday 14 September 2010] [09:57:55] <cremes> no need to encode the subscription topic unless you want to decode it on each subscriber too
[Tuesday 14 September 2010] [09:59:20] <Tasser> hmm
[Tuesday 14 September 2010] [10:00:50] <cremes> hmm sometimes means “i don’t know what you are talking about” :)
[Tuesday 14 September 2010] [10:10:21] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [10:51:56] Quit bgranger has left this server (Client Quit).
[Tuesday 14 September 2010] [10:58:00] <Tasser> cremes, well, so you could filter based on JSON, but that would make it bloat
[Tuesday 14 September 2010] [10:58:28] Join jond has joined this channel (
~jon@87.74.16.24).
[Tuesday 14 September 2010] [10:59:41] <cremes> Tasser: my point is that 0mq matches topics by comparing bytes
[Tuesday 14 September 2010] [10:59:48] <cremes> so you want it to fail as soon as possible
[Tuesday 14 September 2010] [11:00:05] <cremes> using a json-encoded topic means that more bytes would need to be compared to find mismatches
[Tuesday 14 September 2010] [11:00:13] <cremes> so subscription filtering would be slower
[Tuesday 14 September 2010] [11:00:23] <cremes> for example...
[Tuesday 14 September 2010] [11:00:51] <cremes> sock.setsockopt ZMQ::SUBSCRIBE, ‘topic.string’
[Tuesday 14 September 2010] [11:00:53] <cremes> versus
[Tuesday 14 September 2010] [11:01:25] <cremes> sock.setsockopt(ZMQ::SUBSCRIBE, JSON.encode({‘topic’ => ‘topic.string’})
[Tuesday 14 September 2010] [11:01:36] <cremes> honestly, i’m not even sure if that second example would ever work
[Tuesday 14 September 2010] [11:01:53] <cremes> since the json encoder could put things in different order while still producing valid json
[Tuesday 14 September 2010] [11:02:10] <cremes> just use a string for the topic and put your encoded data into a second message part
[Tuesday 14 September 2010] [11:02:48] <Tasser> yeah, probably the sane idea of using it.
[Tuesday 14 September 2010] [11:50:19] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Tuesday 14 September 2010] [11:53:31] Quit zoobab has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [12:05:40] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [12:11:28] Quit jdknezek has left this server (Quit: Page closed).
[Tuesday 14 September 2010] [12:36:34] Quit
keffo_ has left this server (Ping timeout: 265 seconds).
[Tuesday 14 September 2010] [13:00:39] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Tuesday 14 September 2010] [13:01:00] <kleppari> hello
[Tuesday 14 September 2010] [13:02:38] <ModusPwnens> Im not sure, but if you do register, let me know if you are successful. I was trying to the other day and it wouldn’t work =/
[Tuesday 14 September 2010] [13:03:01] <pieterh> kleppari: isn’t is showing? rats... :-( thanks for pointing it out
[Tuesday 14 September 2010] [13:03:08] <guido_g> f*ck, this is wikidot crap
[Tuesday 14 September 2010] [13:03:14] <guido_g> sorry
[Tuesday 14 September 2010] [13:03:32] <pieterh> gudio_g: default permissions on cloned site are private, I forgot
[Tuesday 14 September 2010] [13:03:54] <guido_g> and you need to allow cross-domain cookies
[Tuesday 14 September 2010] [13:03:57] <pieterh> happily it’s a 10-second fix
[Tuesday 14 September 2010] [13:04:09] <guido_g> wikidot sucks in this regard
[Tuesday 14 September 2010] [13:04:10] <pieterh> guido_g: you need to not disallow the dratted things
[Tuesday 14 September 2010] [13:04:21] <pieterh> it’s for good security reasons, unfortunately
[Tuesday 14 September 2010] [13:04:32] <kleppari> registered with wikidots, worked after that. I think it’s a major fail to require it though :)
[Tuesday 14 September 2010] [13:04:43] <pieterh> kleppari: not required, it was my fault
[Tuesday 14 September 2010] [13:04:44] <kleppari> pieterh: np, thanks for the guide :)
[Tuesday 14 September 2010] [13:04:49] <guido_g> pieterh: cross-donmain cookies are for security?
[Tuesday 14 September 2010] [13:04:50] <pieterh> it should work for anonymous users now
[Tuesday 14 September 2010] [13:05:08] <pieterh> guido_g: wikidot splits off insecure stuff (file uploads) to a separate domain
[Tuesday 14 September 2010] [13:05:08] <guido_g> nack
[Tuesday 14 September 2010] [13:05:30] <guido_g> pieterh: it doesen’t work
[Tuesday 14 September 2010] [13:05:30] <pieterh> feel free to debate with the wikidot devs, they are pretty sure of their skills in this respect
[Tuesday 14 September 2010] [13:05:46] <pieterh> what doesn’t work?
[Tuesday 14 September 2010] [13:05:48] <kleppari> pieterh: works now, thanks.
[Tuesday 14 September 2010] [13:05:52] <pieterh> ?
[Tuesday 14 September 2010] [13:06:03] <kleppari> i.e., anonymous users on zguide.zeromq.org
[Tuesday 14 September 2010] [13:06:11] <kleppari> oh, no, sorry
[Tuesday 14 September 2010] [13:06:14] <guido_g> w/ the damn cookies
[Tuesday 14 September 2010] [13:06:51] <pieterh> guido_g: anything different than zeromq.org?
[Tuesday 14 September 2010] [13:07:03] <pieterh> there’s nothing special about this site afaics
[Tuesday 14 September 2010] [13:07:06] <Samy> sustrik, ping?
[Tuesday 14 September 2010] [13:07:21] <guido_g> i allowed the cookies for zeromq.org and the wikidot server
[Tuesday 14 September 2010] [13:07:49] <guido_g> but i’m not going to do that for every new site
[Tuesday 14 September 2010] [13:07:55] <pieterh> ah, well, allow them for
*.zeromq.org and
*.zeromq.com :-)
[Tuesday 14 September 2010] [13:08:08] <pieterh> why do you disallow them?
[Tuesday 14 September 2010] [13:08:18] <guido_g> i do not allow cookies generally
[Tuesday 14 September 2010] [13:08:26] <pieterh> well...
[Tuesday 14 September 2010] [13:08:48] <pieterh> kleppari: thanks, I’ll try it...
[Tuesday 14 September 2010] [13:08:51] * pieterh recalls people saying “I don’t use JavaScript”
[Tuesday 14 September 2010] [13:09:14] <pieterh> unfortunately interactive websites kind of need stuff like that
[Tuesday 14 September 2010] [13:09:43] <guido_g> viewing a page is not interactive
[Tuesday 14 September 2010] [13:10:08] <kleppari> sorry, screwed up file ownership. the screenshot of unauthenticated zguide.zeromq.org should be ok now.
[Tuesday 14 September 2010] [13:10:41] <pieterh> kleppari, hang on, 30 seconds... I’ll fix it, thanks for the screenshot
[Tuesday 14 September 2010] [13:11:10] <pieterh> can you try to reload the site please?
[Tuesday 14 September 2010] [13:11:13] <pieterh> ctrl-R
[Tuesday 14 September 2010] [13:11:23] <kleppari> like a glove
[Tuesday 14 September 2010] [13:11:25] <kleppari> thanks :)
[Tuesday 14 September 2010] [13:11:39] <pieterh> i didn’t change anything, it was already working... :-)
[Tuesday 14 September 2010] [13:11:50] <pieterh> since 19:03 CET
[Tuesday 14 September 2010] [13:12:10] <pieterh> ok, kleppari I appreciate the heads up, kind of stupid to have a Guide no-one can read... :-)
[Tuesday 14 September 2010] [13:12:26] <kleppari> screenshot snapped at 17:06 GMT. Weird.
[Tuesday 14 September 2010] [13:12:32] <kleppari> you’re welcome
[Tuesday 14 September 2010] [13:12:56] <pieterh> guido_g, yeah, because Sustrik pointed out that parts of it were bogus
[Tuesday 14 September 2010] [13:13:00] <pieterh> i’m rewriting it
[Tuesday 14 September 2010] [13:13:05] <pieterh> as soon as i finish my pasta and wine
[Tuesday 14 September 2010] [13:13:09] <guido_g> and why this intermediate page when i click in “The Guide” from zeromq.org?
[Tuesday 14 September 2010] [13:13:37] <pieterh> you’d rather leap to chapter 1 right away?
[Tuesday 14 September 2010] [13:13:48] <guido_g> yes
[Tuesday 14 September 2010] [13:13:55] <pieterh> i mean when you get a book, you see the cover first...
[Tuesday 14 September 2010] [13:14:07] <pieterh> but ok, can be done...
[Tuesday 14 September 2010] [13:14:08] <guido_g> the link is named “The Guide”
[Tuesday 14 September 2010] [13:14:24] <pieterh> Yeah, not The Guide Chapter 1 :-)
[Tuesday 14 September 2010] [13:14:36] <guido_g> not “promotional intermediate page w/o content”
[Tuesday 14 September 2010] [13:14:54] <pieterh> hey, don’t mock the promotional pages...
[Tuesday 14 September 2010] [13:15:22] <guido_g> only if you reduce the size of the MQ logo drastically
[Tuesday 14 September 2010] [13:15:25] <pieterh> delayed gratification...
[Tuesday 14 September 2010] [13:15:40] <pieterh> OMG don’t touch the logo, it’s sacred...
[Tuesday 14 September 2010] [13:15:44] <guido_g> btw, too much red makes aggressive :)
[Tuesday 14 September 2010] [13:15:53] <pieterh> ok, the link from the sidebad should now go straight to the first chapter
[Tuesday 14 September 2010] [13:16:20] <guido_g> nope
[Tuesday 14 September 2010] [13:16:33] * pieterh is on a lousy 2G modem that takes ages
[Tuesday 14 September 2010] [13:16:55] <guido_g> ok, i’ll wait
[Tuesday 14 September 2010] [13:17:20] <pieterh> guido_g, when you say “nope” you mean as in “oops, I didn’t reload it and HTTP doesn’t yet have a autoupdate function”...?
[Tuesday 14 September 2010] [13:17:57] <guido_g> pieterh: no, it means “i pressed relaod like maniac but the thing didn’t work”
[Tuesday 14 September 2010] [13:18:09] <pieterh> works for me...
[Tuesday 14 September 2010] [13:18:22] <pieterh> go back to www.zeromq.org
[Tuesday 14 September 2010] [13:18:46] Quit speedy1 has left this server (Quit: Leaving).
[Tuesday 14 September 2010] [13:18:47] <pieterh> are there more links? I fixed the one in the sidebar
[Tuesday 14 September 2010] [13:18:51] <pieterh> what one are you clicking on?
[Tuesday 14 September 2010] [13:18:54] <guido_g> i am on zeromq.org and open the link in another tab
[Tuesday 14 September 2010] [13:19:04] <guido_g> after reloading the page
[Tuesday 14 September 2010] [13:19:49] <pieterh> shrug...it’s correct... you are reloading zeromq.org?
[Tuesday 14 September 2010] [13:19:57] <guido_g> yes
[Tuesday 14 September 2010] [13:20:16] <guido_g> hmmm... cdn thing maybe
[Tuesday 14 September 2010] [13:20:25] <pieterh> did you sacrifice a chicken to the gnomes of the Internet?
[Tuesday 14 September 2010] [13:20:53] <guido_g> ahhh... i constantly miss this one (eating the chicken myself)
[Tuesday 14 September 2010] [13:21:07] <pieterh> might be some caching somewhere your side of things
[Tuesday 14 September 2010] [13:21:19] <pieterh> it’s correct, anyhow, just tested again
[Tuesday 14 September 2010] [13:21:31] <guido_g> ahhh... cdn got the update
[Tuesday 14 September 2010] [13:21:35] <kleppari> you might also be clicking different links?
[Tuesday 14 September 2010] [13:21:40] <guido_g> no
[Tuesday 14 September 2010] [13:22:09] <kleppari> the sidebar link points to chapter one, the other guide links point to the ‘promotional’ page
[Tuesday 14 September 2010] [13:22:27] * pieterh looks for the other guide links
[Tuesday 14 September 2010] [13:22:29] <guido_g> so remember, it might take approx. 5 minutes til changes become vsisible
[Tuesday 14 September 2010] [13:22:41] <kleppari> two of them, one under most popular, other in recent site changes
[Tuesday 14 September 2010] [13:22:51] <pieterh> ah, kleppari, right
[Tuesday 14 September 2010] [13:23:19] <kleppari> but.. After a little while I like the promotional page
[Tuesday 14 September 2010] [13:23:28] <pieterh> well, indeed, a book deserves a cover
[Tuesday 14 September 2010] [13:23:40] <pieterh> bright red
[Tuesday 14 September 2010] [13:24:17] <pieterh> anyhow, this is always a work in progress, i like to see how people respond over time before making more changes
[Tuesday 14 September 2010] [13:24:33] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 14 September 2010] [13:24:48] <kleppari> I didn’t get it at first though, started at zeromq.com -> hit read the manual -> clicked ‘MQ - The guide’ -> got to zguide.zeromq.org
[Tuesday 14 September 2010] [13:24:51] <guido_g> pieterh: you’re on a laptop?
[Tuesday 14 September 2010] [13:25:01] <kleppari> didn’t catch the differences between zguide.zeromq.org and zeromq.com right away
[Tuesday 14 September 2010] [13:25:13] <pieterh> guido_g, on a laptop with a 2G phone modem, eating pasta in a hotel
[Tuesday 14 September 2010] [13:25:26] <guido_g> it’S more about the display size
[Tuesday 14 September 2010] [13:25:32] <kleppari> sounds like a crap hotel :P
[Tuesday 14 September 2010] [13:25:53] <pieterh> kleppari, it’s a pretty amazing hotel actually, an old Soviet relic
[Tuesday 14 September 2010] [13:26:04] <pieterh> guido_g, yes, laptop display
[Tuesday 14 September 2010] [13:26:38] <guido_g> pieterh: because on my 13” display the logo takes up apporx. 1/3 of the vertical space
[Tuesday 14 September 2010] [13:26:51] <guido_g> which is way to much, imho
[Tuesday 14 September 2010] [13:27:08] <guido_g> worse if i zoom in for better readability
[Tuesday 14 September 2010] [13:27:18] <pieterh> guido_g, yes, on mine too but it doesn’t matter on the cover page, and hardly matters on the content pages since you need to scroll anyhow
[Tuesday 14 September 2010] [13:27:33] <kleppari> looks good on 1280x1024
[Tuesday 14 September 2010] [13:27:42] <pieterh> we tried using a smaller logo some time ago, people felt it wasn’t nice
[Tuesday 14 September 2010] [13:28:01] <kleppari> pieterh: and no rj45 jack? URL? :)
[Tuesday 14 September 2010] [13:28:20] <guido_g> ok, i give up
[Tuesday 14 September 2010] [13:28:27] <pieterh> kleppari, URL? no rj45, no, wifi but it’s too slow
[Tuesday 14 September 2010] [13:28:48] <kleppari> url of the hotel website? or is it too much of a relic?
[Tuesday 14 September 2010] [13:28:49] <pieterh> guido_g, do start a discussion on look and feel, but this isn’t the place IMO
[Tuesday 14 September 2010] [13:29:23] <pieterh> guido_g, the original look was designed by a proper artist, and anything that replaces it would have to be done carefully
[Tuesday 14 September 2010] [13:29:32] <pieterh> kleppari, it’s the Hotel Kyjev in Bratislava
[Tuesday 14 September 2010] [13:30:38] <guido_g> pieterh: i didn’t complain about the logo, only it’s size, but nevermind i’ll find a solution
[Tuesday 14 September 2010] [13:30:57] <pieterh> guido_g, I do agree the logo is HUGE but my previous attempts to downsize it failed
[Tuesday 14 September 2010] [13:31:23] <guido_g> ok
[Tuesday 14 September 2010] [13:32:11] <pieterh> guido_g, what might work is a lighter theme for sites other than the Start Here site
[Tuesday 14 September 2010] [13:32:23] <pieterh> now that we have more sites, we can fine-tune them
[Tuesday 14 September 2010] [13:38:04] <guido_g> except for the “promo” pages the focus should be on readability
[Tuesday 14 September 2010] [13:42:05] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [13:53:08] Quit
pieterh_ has left this server (Read error: No route to host).
[Tuesday 14 September 2010] [14:52:29] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [14:54:31] Quit xla has left this server (Ping timeout: 255 seconds).
[Tuesday 14 September 2010] [15:29:26] Quit
pieterh__ has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [16:18:53] Quit benoitc has left this server (Ping timeout: 245 seconds).
*** on Tue Sep 14 16:37:22 2010
[Tuesday 14 September 2010] [16:37:22] Topic The channel topic is “Welcome!”.
[Tuesday 14 September 2010] [16:37:28] Mode Channel modes: no messages from outside, topic protection
[Tuesday 14 September 2010] [16:37:28] Created This channel was created on 2010-02-05 10:44.
[Tuesday 14 September 2010] [16:44:44] Part bobs has left this channel.
[Tuesday 14 September 2010] [17:10:43] Quit rphillips has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [18:06:56] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [18:43:28] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [18:57:59] Quit plq has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [19:40:23] Quit bgranger has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [19:44:27] Quit
bgranger_ has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [20:42:43] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [21:50:01] <kenkeiter> Okay, Rubyists with zmq experience – question!
[Tuesday 14 September 2010] [21:55:01] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [21:55:09] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [22:20:34] Quit AlexB has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [22:20:38] Join AlexB has joined this channel (
alex@brasetvik.com).
[Tuesday 14 September 2010] [22:23:00] <
Xin_> test
[Tuesday 14 September 2010] [22:25:21] <
Xin_> Does it make sense to add UDP support to 0mq?
[Tuesday 14 September 2010] [22:27:20] Quit
Xin_ has left this server (Quit: Page closed).
[Tuesday 14 September 2010] [22:47:48] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Tuesday 14 September 2010] [22:56:32] Quit kenkeiter has left this server (Quit: kenkeiter).
[Tuesday 14 September 2010] [22:57:34] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [23:27:41] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [23:43:31] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [00:03:41] Quit rgl has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [00:54:36] Join kenkeiter has joined this channel (
~kenkeiter@2002:4c73:685:1234:223:6cff:fe98:be3b).
[Wednesday 15 September 2010] [00:54:47] Quit kenkeiter has left this server (Client Quit).
[Wednesday 15 September 2010] [01:16:01] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 15 September 2010] [01:17:36] Quit guido_g has left this server (Client Quit).
[Wednesday 15 September 2010] [01:32:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [02:23:26] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [02:24:37] Quit bgranger has left this server (Client Quit).
[Wednesday 15 September 2010] [02:26:58] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 15 September 2010] [02:31:02] Quit keffo has left this server (Ping timeout: 272 seconds).
[Wednesday 15 September 2010] [02:47:24] Quit carrus85 has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [03:02:03] Quit kleppari has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [03:04:21] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [03:10:47] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Wednesday 15 September 2010] [03:15:41] Quit
AndrewBC_ has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [03:22:45] Quit plq has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [03:32:39] Quit rgl has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [04:06:06] <sustrik> Samy: pong
[Wednesday 15 September 2010] [04:06:38] <Zao> Doesn’t PGM stuff travel on UDP already?
[Wednesday 15 September 2010] [04:06:48] * Zao notes the six hour gap in time.
[Wednesday 15 September 2010] [04:09:05] <sustrik> Zao: PGM travels on IP
[Wednesday 15 September 2010] [04:09:22] <sustrik> epgm is a convenience hack that travels on UDP instead of IP
[Wednesday 15 September 2010] [04:09:32] <Zao> Oh, right.
[Wednesday 15 September 2010] [04:09:43] <Zao> Never used any of them, so I went with faint memories of manpages past.
[Wednesday 15 September 2010] [04:11:43] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Wednesday 15 September 2010] [04:53:30] Quit manveru has left this server (Quit: WeeChat 0.3.2).
[Wednesday 15 September 2010] [05:15:54] Join omarkj has joined this channel (
~omarkj@81.15.34.139).
[Wednesday 15 September 2010] [05:18:46] <omarkj> kleppari: Morning.
[Wednesday 15 September 2010] [05:28:58] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [05:29:15] <mrm2m> Hey ho!
[Wednesday 15 September 2010] [05:30:25] <mrm2m> Is there a lightweight introduction, how zmq works internally?
[Wednesday 15 September 2010] [05:33:01] <pieterh> mrm2m: there is documentation of the source code and there are whitepapers
[Wednesday 15 September 2010] [05:33:11] <pieterh> but there is nothing lightweight about 0MQ’s internals, really
[Wednesday 15 September 2010] [05:34:06] <mrm2m> ok
[Wednesday 15 September 2010] [05:39:21] <Bruc> hey wat sup all
[Wednesday 15 September 2010] [05:39:51] Part Bruc has left this channel.
[Wednesday 15 September 2010] [05:54:50] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [06:00:48] <kleppari> omarkj: hah, hi!
[Wednesday 15 September 2010] [06:01:15] <omarkj> kleppari: Didn’t notice you here until now, are you guys using 0mq?
[Wednesday 15 September 2010] [06:01:37] <kleppari> I’m experimenting a bit
[Wednesday 15 September 2010] [06:02:23] <kleppari> I just love the idea of message passing for concurrency
[Wednesday 15 September 2010] [06:03:08] <kleppari> well, outside of erlang at least
[Wednesday 15 September 2010] [06:05:14] <kleppari> how about yourself?
[Wednesday 15 September 2010] [06:07:23] <omarkj> Working with it.
[Wednesday 15 September 2010] [06:07:32] <omarkj> Let’s go pm. :)
[Wednesday 15 September 2010] [06:35:00] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [06:54:46] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [07:21:07] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [08:05:56] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [08:29:01] Quit jsimmons has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [08:49:08] <keffo> sustrik, I think I might have figured out the concurrency issue
[Wednesday 15 September 2010] [08:49:38] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [08:49:44] <sustrik> cremes: here i am
[Wednesday 15 September 2010] [08:50:00] <sustrik> keffo: so what’s the problem?
[Wednesday 15 September 2010] [08:51:59] <keffo> I was rewriting (using poll rather than sequential) and to clean up logging etc, and I just noticed one place where I send a null message before sending the rest, except “the rest” already included the incomin route-null, so I suspect it was being sent twie
[Wednesday 15 September 2010] [08:51:59] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [08:52:00] <keffo> twice
[Wednesday 15 September 2010] [08:52:40] <keffo> nullmore, nullmore, uuidmore, data, etc..
[Wednesday 15 September 2010] [08:52:52] <sustrik> ok, i see
[Wednesday 15 September 2010] [08:53:05] <sustrik> so no problem in 0mq to worry about
[Wednesday 15 September 2010] [08:53:10] <keffo> Not at all
[Wednesday 15 September 2010] [08:53:15] <sustrik> :)
[Wednesday 15 September 2010] [08:53:22] <keffo> I never expected that either :)
[Wednesday 15 September 2010] [08:53:48] <sustrik> you are an optimist then :)
[Wednesday 15 September 2010] [08:54:23] <keffo> It’s been very stable and well behaved so far, apart from this issue which is my own fault :)
[Wednesday 15 September 2010] [08:55:46] <keffo> I did 100k pi calcs yesterday, with 10 worker procs on different machines.. the loadbalancer had ~500kb/s passing through it, and in total more than half a gig of logs were generated.. That’s when I decided the logging was way wonky, didnt really tell much
[Wednesday 15 September 2010] [09:01:49] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 15 September 2010] [09:25:04] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [09:25:41] <cremes> sustrik: i’m available for the next hour or so if you want to chat
[Wednesday 15 September 2010] [09:32:23] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [09:45:14] <sustrik> cremes: hi
[Wednesday 15 September 2010] [09:46:04] <sustrik> give me some context first: why have you moved from ruby-ffi to rbzmq?
[Wednesday 15 September 2010] [09:46:49] <cremes> there are multiple ruby runtimes; not all of them support the C extension api that rbzmq uses
[Wednesday 15 September 2010] [09:47:00] <cremes> ffi allows me to support all of the runtimes
[Wednesday 15 September 2010] [09:47:38] <cremes> oh, and i haven’t moved from one to the other; i would just like rbzmq to have the same api as ffi-rzmq
[Wednesday 15 September 2010] [09:48:13] <sustrik> ok, i see
[Wednesday 15 September 2010] [09:48:27] <sustrik> it looks like brian buchanan is not reponding, right?
[Wednesday 15 September 2010] [09:48:39] <cremes> right
[Wednesday 15 September 2010] [09:48:48] <sustrik> ok, i have admin access to the project
[Wednesday 15 September 2010] [09:49:02] <sustrik> i’ll add you there as a developer
[Wednesday 15 September 2010] [09:49:10] <mato> sustrik: cremes: a question; i was wondering myself why there were two ruby bindings
[Wednesday 15 September 2010] [09:49:16] <mato> is there any point in keeping both around?
[Wednesday 15 September 2010] [09:49:20] <sustrik> hi
[Wednesday 15 September 2010] [09:49:37] <mato> if FFI is the way to go then why keep rbzmq at all?
[Wednesday 15 September 2010] [09:50:38] <cremes> um... i don’t know; it’s probably best to get feedback from folks using rbzmq and ask them why they prefer it
[Wednesday 15 September 2010] [09:50:53] <cremes> i’m biased towards the ffi one, obviously ;)
[Wednesday 15 September 2010] [09:51:01] <mato> that might be a good idea; having two bindings is IMO confusing
[Wednesday 15 September 2010] [09:51:03] <sustrik> does it work with any runtime?
[Wednesday 15 September 2010] [09:51:14] <cremes> that’s why i was hoping brian buchanan would speak up
[Wednesday 15 September 2010] [09:51:43] <mato> cremes: is there a mailing list for this? I’ve not seen any discussion on zeromq-dev...
[Wednesday 15 September 2010] [09:52:18] <mato> maybe just ping there, outline the situation (two bindings, ffi works on all runtimes, etc.) and ask the community
[Wednesday 15 September 2010] [09:52:21] <cremes> this has not been brought up on the zerome-dev list
[Wednesday 15 September 2010] [09:52:24] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Wednesday 15 September 2010] [09:52:39] <cremes> i’ll do that
[Wednesday 15 September 2010] [09:53:01] <sustrik> can you explain how does the runtime interoperability work?
[Wednesday 15 September 2010] [09:53:24] <sustrik> so far i had an impression that ffi doesn’t work with all the ruby runtimes
[Wednesday 15 September 2010] [09:53:43] <cremes> sustrik: as of a few days ago, ffi works with all of the runtimes
[Wednesday 15 September 2010] [09:53:48] <sustrik> aha
[Wednesday 15 September 2010] [09:53:50] <cremes> i worked out the last details with the runtime guys
[Wednesday 15 September 2010] [09:53:57] <sustrik> so it’s new
[Wednesday 15 September 2010] [09:54:09] <cremes> the issue with C extensions is that they oftentimes access lots of internal memory structures
[Wednesday 15 September 2010] [09:54:09] <sustrik> that may actually mean that rbzmq is not needed any more
[Wednesday 15 September 2010] [09:54:57] <mato> cremes: what’s the status of FFI support for Ruby MRI (i.e. that which most people are using right now)?
[Wednesday 15 September 2010] [09:54:59] <cremes> ffi allows the runtime authors to hide their implementation details and provide a more solid mechanism for accessing C libraries
[Wednesday 15 September 2010] [09:55:06] <sustrik> i see
[Wednesday 15 September 2010] [09:55:08] <cremes> mato: it’s quite good
[Wednesday 15 September 2010] [09:55:26] <mato> cremes: right, but does “quite good” mean “production quality”?
[Wednesday 15 September 2010] [09:55:57] <mato> i guess what i’m asking is if rbzmq goes away then what will that mean for the current mainstream of Ruby MRI users
[Wednesday 15 September 2010] [09:56:11] <mato> will they all just happily start using the FFI binding, or not?
[Wednesday 15 September 2010] [09:56:12] <cremes> mato: perhaps; i can only speak for how it works with 0mq; obviously we still have hangs until 2.1.x comes out with EINTR support for blocking calls
[Wednesday 15 September 2010] [09:56:39] <mato> sure, but the EINTR hangs are orthogonal to FFI/non-FFI
[Wednesday 15 September 2010] [09:56:41] <sustrik> well, whatever happens the rbzmq project is not going to be killed
[Wednesday 15 September 2010] [09:56:44] <sustrik> so no worry
[Wednesday 15 September 2010] [09:56:46] <cremes> mato: i’ll ask for feedback on the ML
[Wednesday 15 September 2010] [09:57:04] <sustrik> the problem is that it doesn’t have a permanent maintainer
[Wednesday 15 September 2010] [09:57:18] <cremes> mato: sure, it’s orthogonal, but it’s hard to say if there are other bugs when you get a hang
[Wednesday 15 September 2010] [09:57:25] <cremes> is it due to the blocking behavior or something else?
[Wednesday 15 September 2010] [09:57:31] <mato> well then put a note up about that (lack of maintainer) on the rbzmq project page
[Wednesday 15 September 2010] [09:57:39] <cremes> i don’t have the time or energy to run stuff under gdb all of the time to figure that out :)
[Wednesday 15 September 2010] [09:58:07] <cremes> let’s take this to the ML and see what users have to say
[Wednesday 15 September 2010] [09:58:10] <mato> yup
[Wednesday 15 September 2010] [09:58:12] <sustrik> ok
[Wednesday 15 September 2010] [09:58:18] <cremes> btw, from a search on github most projects are using the ffi-based binding
[Wednesday 15 September 2010] [09:58:36] <cremes> but i certainly don’t want to lose or alienate the rbzmq users
[Wednesday 15 September 2010] [09:59:01] <cremes> which takes us full circle; i’d like each one to present the same api to the user (based off of the C and C++ binding apis)
[Wednesday 15 September 2010] [09:59:11] <mato> definitely
[Wednesday 15 September 2010] [09:59:22] <mato> otherwise it just all gets way too confusing
[Wednesday 15 September 2010] [09:59:26] <cremes> if people can switch back and forth with no code changes, that’s a great situation for the community
[Wednesday 15 September 2010] [09:59:33] <mato> is it much work to synchronize the APIs?
[Wednesday 15 September 2010] [10:00:02] <cremes> mato: not a lot but unfortunately i don’t have the C chops to do it myself
[Wednesday 15 September 2010] [10:00:12] <cremes> so i need someone else to agree to it
[Wednesday 15 September 2010] [10:00:17] <cremes> to do the work
[Wednesday 15 September 2010] [10:00:42] <mato> cremes: well, ask for help on the list and we’ll see what happens
[Wednesday 15 September 2010] [10:00:53] <cremes> will do
[Wednesday 15 September 2010] [10:01:08] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [10:05:30] <sustrik> mato: man bug!
[Wednesday 15 September 2010] [10:05:44] <sustrik> zmq_socket(3):
[Wednesday 15 September 2010] [10:05:47] <sustrik> “When a ZMQ_XREQ socket is connected to a ZMQ_REP socket each message sent must consist of an empty message part, the delimiter, followed by one or more body parts.”
[Wednesday 15 September 2010] [10:05:48] <mato> sustrik: !!?!? :-)
[Wednesday 15 September 2010] [10:06:22] <sustrik> the identities should be mentioned
[Wednesday 15 September 2010] [10:06:48] <sustrik> they are definitely mentioned for ZMQ_XREP
[Wednesday 15 September 2010] [10:06:52] <ptrb> yeah, it’s not really clear to me what work i have to do at the app level to do XREQ/REP (or REQ/XREP)
[Wednesday 15 September 2010] [10:07:09] <mato> uh, yeah, i kind of punted on explaining the identity stack on the XREQ side at the time
[Wednesday 15 September 2010] [10:07:20] <mato> sustrik: that stuff was written in a hurry, you remember :)
[Wednesday 15 September 2010] [10:07:23] <sustrik> ptrb: there was a nice diagram somewhere...
[Wednesday 15 September 2010] [10:07:39] <mato> sustrik: anyway, it’s not a bug, it’s just a simplification :)
[Wednesday 15 September 2010] [10:07:43] <mato> sustrik: i’ll look into it.
[Wednesday 15 September 2010] [10:08:39] <cremes> if not, tell me what is unclear and i’ll fix it (or edit the page yourself)
[Wednesday 15 September 2010] [10:08:40] <ptrb> clicks
[Wednesday 15 September 2010] [10:08:52] <sustrik> ah – that’s the one i meant
[Wednesday 15 September 2010] [10:08:55] <cremes> i was certainly confused by that stuff which is why i wrote it down
[Wednesday 15 September 2010] [10:09:14] <ptrb> yeah, it definitely helps
[Wednesday 15 September 2010] [10:09:23] <ptrb> but ruby is definitely not the right language to use in examples like this
[Wednesday 15 September 2010] [10:09:47] <ptrb> it’s as opaque as perl without the benefit of perl’s saturation
[Wednesday 15 September 2010] [10:09:53] <cremes> ha
[Wednesday 15 September 2010] [10:10:03] <mato> :-)
[Wednesday 15 September 2010] [10:10:37] * mato will think of some suitably terse way of describing the identity stack in zmq_socket(3)
[Wednesday 15 September 2010] [10:36:52] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 15 September 2010] [10:45:23] <CIA-20> zeromq2: 03Martin Sustrik 07maint * re2802d9 10/ src/options.cpp : values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9lAWvv[Wednesday 15 September 2010] [10:46:54] <pieterh> sustrik: nice to see the validation of option values
[Wednesday 15 September 2010] [10:47:45] <sustrik> there are 3 bugs filled by cremes :)
[Wednesday 15 September 2010] [10:47:53] <mato> sustrik: incidentally, there’s one annoying thing regarding the option values
[Wednesday 15 September 2010] [10:48:00] <sustrik> yes?
[Wednesday 15 September 2010] [10:48:14] <mato> well, it’s the whole “option types” issue
[Wednesday 15 September 2010] [10:48:24] <sustrik> ?
[Wednesday 15 September 2010] [10:48:26] <mato> one thing that i realised the other day when i was writing some simple test cases
[Wednesday 15 September 2010] [10:48:47] <mato> is that in order to use options sensibly from C, you have to have uint64_t / int64_t defined
[Wednesday 15 September 2010] [10:49:03] <mato> now, obviously those are not defined by zmq.h
[Wednesday 15 September 2010] [10:49:25] <mato> of course if you want to use the API you have to do “platform specific stuff” to get those types
[Wednesday 15 September 2010] [10:49:31] <mato> not a good situation :-(
[Wednesday 15 September 2010] [10:50:27] <mato> especially on windows where M$ does not ship stdint.h
[Wednesday 15 September 2010] [10:50:36] <sustrik> i know
[Wednesday 15 September 2010] [10:50:54] <sustrik> what can i do without breaking the backward compatibility?
[Wednesday 15 September 2010] [10:51:15] <mato> probably nothing
[Wednesday 15 September 2010] [10:51:20] <sustrik> :|
[Wednesday 15 September 2010] [10:51:54] <mato> hmm, maybe...
[Wednesday 15 September 2010] [10:52:37] <mato> well, in theory you could define
zmq_ namespaced equivalents of those types including the platform magic in zmq.h, but that’s kind of ugly
[Wednesday 15 September 2010] [10:52:52] <mato> and it’d probably still break C++ due to it’s strict notions of types
[Wednesday 15 September 2010] [10:52:56] <sustrik> it has to be solved properly
[Wednesday 15 September 2010] [10:53:04] <sustrik> BSD-style
[Wednesday 15 September 2010] [10:53:13] <mato> hmm
[Wednesday 15 September 2010] [10:53:17] <sustrik> 4-byte unsinged integer in network byte order
[Wednesday 15 September 2010] [10:53:20] <mato> hang on
[Wednesday 15 September 2010] [10:53:21] <sustrik> and such
[Wednesday 15 September 2010] [10:53:22] <mato> what?
[Wednesday 15 September 2010] [10:53:30] <mato> what does network byte order have to do with it? :)
[Wednesday 15 September 2010] [10:53:45] <sustrik> that’s what i do when usign BSD sockets
[Wednesday 15 September 2010] [10:53:52] <mato> ?
[Wednesday 15 September 2010] [10:54:10] <sustrik> addr.port = htons (port);
[Wednesday 15 September 2010] [10:55:56] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [10:58:19] <mato> sustrik: ah, you’re right, i didn’t realise e.g. sockaddr_in.sin_port was in network byte order
[Wednesday 15 September 2010] [10:58:32] <mato> sustrik: the thing is, the setsockopt stuff is somewhat ad-hoc
[Wednesday 15 September 2010] [10:58:43] <CIA-20> zeromq2: 03Martin Sustrik 07master * re2802d9 10/ src/options.cpp : values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9lAWvv[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: 03Martin Sustrik 07master * rff10807 10/ src/options.cpp :
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: * maint:
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9P99Fu[Wednesday 15 September 2010] [10:58:48] <mato> it does not use network byte order AFAIK
[Wednesday 15 September 2010] [10:59:18] <mato> sustrik: anyway thinking about it, i guess zmq.h will have to define its own types
[Wednesday 15 September 2010] [11:00:14] <mato> the main problem with that is doing it in a portable fashion :-(
[Wednesday 15 September 2010] [11:07:22] <ptrb> #defines, #defines for everybody!!
[Wednesday 15 September 2010] [11:07:58] <mato> sure, feel free to suggest how to portably define a 64-bit unsigned integer type :-)
[Wednesday 15 September 2010] [11:08:20] <ptrb> you just take two ints and mash ‘em together, obviously
[Wednesday 15 September 2010] [11:08:27] <mato> :-)
[Wednesday 15 September 2010] [11:29:09] <sustrik> mato: btw, there’s some packaging PATCH on the mailing list
[Wednesday 15 September 2010] [11:29:15] <sustrik> will you reply to that?
[Wednesday 15 September 2010] [11:30:30] <mato> sustrik: yes, later, i know it’s there
[Wednesday 15 September 2010] [11:30:32] <mato> have to go now
[Wednesday 15 September 2010] [11:30:45] <sustrik> ok, there’s on in the bug tracker as well
[Wednesday 15 September 2010] [11:30:51] <mato> sustrik: i’d have just applied it but the license stuff needs to be sorted out
[Wednesday 15 September 2010] [11:31:05] <mato> else all i can reply is “please state your patch is licensed under...”
[Wednesday 15 September 2010] [11:31:14] <mato> have discussed with pieter will update you this evening
[Wednesday 15 September 2010] [11:31:18] <mato> or pieter will
[Wednesday 15 September 2010] [11:31:19] <mato> bbl
[Wednesday 15 September 2010] [11:31:32] <sustrik> should i ask for the license?
[Wednesday 15 September 2010] [11:31:46] <mato> don’t bother lets just fix it
[Wednesday 15 September 2010] [11:31:51] <mato> talk to you in the evening
[Wednesday 15 September 2010] [11:31:54] <sustrik> ok
[Wednesday 15 September 2010] [11:40:49] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [11:41:58] <Samy> sustrik, just had some questions regarding zeromq’s lock-less data structures.
[Wednesday 15 September 2010] [11:42:27] <sustrik> yes?
[Wednesday 15 September 2010] [11:42:51] <Samy> sustrik, I’m working on a library to help ease concurrent programming, it includes a plethora of concurrent data structures, synchronization methods and mechanisms for SMR (currently using hazard pointers, considering implementing RCU).
[Wednesday 15 September 2010] [11:43:37] <Samy> sustrik, I was curious if there were some features you would really like to see, and some specific constrainted data structures (M:N consumer/producers) for zeromq.
[Wednesday 15 September 2010] [11:43:52] <Samy> sustrik, I was also curious if SMR was important for ZeroMQ at all.
[Wednesday 15 September 2010] [11:44:38] <sustrik> sorry, what’s SMR?
[Wednesday 15 September 2010] [11:44:48] <Samy> sustrik, safe memory reclamation.
[Wednesday 15 September 2010] [11:45:26] <Samy> sustrik, usually it’s important for unbunded lock-less data structures (simple stack being a great example).
[Wednesday 15 September 2010] [11:46:48] <sustrik> does it translate to “make the current state of the memory visible to the other CPU codes”?
[Wednesday 15 September 2010] [11:47:23] <sustrik> sorry, i am not an expert on lock-free algos
[Wednesday 15 September 2010] [11:48:03] <Samy> sustrik, well, I guess fundamentally, it’s more like “let other CPUs know I am not using this memory”.
[Wednesday 15 September 2010] [11:48:27] <sustrik> ah, ok, let me explain what 0mq is doing
[Wednesday 15 September 2010] [11:48:28] <Samy> sustrik, some techniques will require a full barrier (RCU) while others do not (hazard pointers).
[Wednesday 15 September 2010] [11:48:41] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [11:48:57] <sustrik> the communication always happens between exactly two endpoins
[Wednesday 15 September 2010] [11:49:03] <sustrik> ie. at most 2 cpu cores
[Wednesday 15 September 2010] [11:49:25] <sustrik> so each lock-free queue has exactly one writer and exactly one reader
[Wednesday 15 September 2010] [11:49:30] <Samy> Ah, makes it much easier. :-)
[Wednesday 15 September 2010] [11:49:36] <Samy> Sorry, let me pop open the source-code.
[Wednesday 15 September 2010] [11:49:48] <sustrik> src/ypipe.hpp
[Wednesday 15 September 2010] [11:49:52] <Samy> Thanks, loading.
[Wednesday 15 September 2010] [11:50:26] <sustrik> what it does basically is that writer is appending new items to the linked list
[Wednesday 15 September 2010] [11:50:39] <sustrik> the reader is reading items from the linked list
[Wednesday 15 September 2010] [11:50:54] <Samy> I don’t see any padding, sustrik.
[Wednesday 15 September 2010] [11:51:08] <sustrik> what padding?
[Wednesday 15 September 2010] [11:51:09] <Samy> Between w and r.
[Wednesday 15 September 2010] [11:51:25] <Samy> sustrik, to prevent false cache line sharing, this improves concurrency.
[Wednesday 15 September 2010] [11:51:36] <sustrik> that would be nice
[Wednesday 15 September 2010] [11:51:49] <Samy> sustrik, it can be drastic. :-)
[Wednesday 15 September 2010] [11:51:53] <Samy> Let me show you a simple example on this machine.
[Wednesday 15 September 2010] [11:52:57] <sustrik> ok, so what has to be done is separate the variables that belong to the reader and those that belong to the writer
[Wednesday 15 September 2010] [11:53:04] <sustrik> and keep them in different cachelines
[Wednesday 15 September 2010] [11:53:06] <sustrik> right?
[Wednesday 15 September 2010] [11:53:27] <keffo> the data too
[Wednesday 15 September 2010] [11:53:48] <Samy> [
sbahra@sbahra validate]$ time ./ck_fifo_spsc 8 1 100000
[Wednesday 15 September 2010] [11:53:49] <Samy> real 0m5.294s
[Wednesday 15 September 2010] [11:53:59] <Samy> [
sbahra@sbahra validate]$ time ./ck_fifo_spsc 8 1 100000
[Wednesday 15 September 2010] [11:53:59] <Samy> real 0m7.718s
[Wednesday 15 September 2010] [11:54:12] <Samy> sustrik, the latter is without appropriate padding.
[Wednesday 15 September 2010] [11:54:16] <sustrik> nice
[Wednesday 15 September 2010] [11:54:26] <sustrik> the data are allocated in a contiguous block
[Wednesday 15 September 2010] [11:54:37] <keffo> that was pretty substantial, what arch is that?
[Wednesday 15 September 2010] [11:54:54] <sustrik> to avoid excessive memory allocation
[Wednesday 15 September 2010] [11:54:59] <Samy> sustrik, that is a simple benchmark that creates a token ring using a single-producer/single-consumer lock-less queue (passes around 100000 across 8 threads some number of iterations)
[Wednesday 15 September 2010] [11:55:05] <Samy> keffo, that is on a Nehalem box.
[Wednesday 15 September 2010] [11:55:09] <keffo> aho
[Wednesday 15 September 2010] [11:55:25] <keffo> sustrik, You should look into pooling, not only for that
[Wednesday 15 September 2010] [11:55:40] <Samy> sustrik, you want the writer and reader variables to be on a seperate cache line.
[Wednesday 15 September 2010] [11:55:56] <sustrik> right, i understand that
[Wednesday 15 September 2010] [11:55:56] <Samy> sustrik, so for example, void
*reader; char pad[56]; void
*writer; ...
[Wednesday 15 September 2010] [11:56:00] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [11:56:12] <sustrik> not sure how to do that in portable fashion though
[Wednesday 15 September 2010] [11:56:22] <keffo> compile time :)
[Wednesday 15 September 2010] [11:56:28] <keffo> macro hell
[Wednesday 15 September 2010] [11:56:29] <ptrb> did someone say #define??
[Wednesday 15 September 2010] [11:56:32] <ptrb> :D
[Wednesday 15 September 2010] [11:56:34] <Samy> sustrik, not very portable (my library tries to make that portable by doing the hard work of generating those constants at compile-time).
[Wednesday 15 September 2010] [11:56:34] <sustrik> :)
[Wednesday 15 September 2010] [11:56:49] <Samy> sustrik, but modern IA32 and SPARCv9 boxes I know of all have 64 byte cache lines.
[Wednesday 15 September 2010] [11:56:57] <sustrik> well, if we assumed the cache line size is 64 bytes
[Wednesday 15 September 2010] [11:56:59] <keffo> cpuid fiddling works I guess :)
[Wednesday 15 September 2010] [11:57:03] <sustrik> we would be right in most cases
[Wednesday 15 September 2010] [11:57:06] <sustrik> no?
[Wednesday 15 September 2010] [11:57:14] <keffo> all x86_64 is >64 no?
[Wednesday 15 September 2010] [11:57:21] <sustrik> 128?
[Wednesday 15 September 2010] [11:57:22] <Samy> keffo, no.
[Wednesday 15 September 2010] [11:57:25] <Samy> keffo, 64.
[Wednesday 15 September 2010] [11:57:38] <Samy> keffo, when I mean IA32, I mean x86/x86_64.
[Wednesday 15 September 2010] [11:57:41] <keffo> >=64 then :)
[Wednesday 15 September 2010] [11:57:47] <Samy> I don’t know of any > 64.
[Wednesday 15 September 2010] [11:58:18] <sustrik> padding to 64 bytes seems reasonable imo
[Wednesday 15 September 2010] [11:58:34] <keffo> I meant, x86_64 is presumably >=64, while x86 is not
[Wednesday 15 September 2010] [11:58:50] <sustrik> that would be 64 bytes per reader and 64 bytes per writer
[Wednesday 15 September 2010] [11:59:06] <sustrik> the shared varaible should probably be on a separate cahceline
[Wednesday 15 September 2010] [11:59:08] <keffo> Samy, Have you tried measuring on itanium? :)
[Wednesday 15 September 2010] [11:59:16] <sustrik> so it’s 192 bytes per queue
[Wednesday 15 September 2010] [11:59:40] <Samy> keffo, Itanium is very interesting, but barely used, so no.
[Wednesday 15 September 2010] [12:00:06] <Samy> keffo, I would like to work on a port of my library there but I have had design issues, just recently did a major design change.
[Wednesday 15 September 2010] [12:00:07] <keffo> Yeah, void of any cachemisses it should be
[Wednesday 15 September 2010] [12:00:20] <Samy> sustrik, sorry, can you continue with your explanation of ypipe?
[Wednesday 15 September 2010] [12:00:25] <sustrik> ah
[Wednesday 15 September 2010] [12:00:42] * keffo butts out :)
[Wednesday 15 September 2010] [12:00:42] <sustrik> ok, so part of the list belongs to the writer thread
[Wednesday 15 September 2010] [12:00:50] <sustrik> part of it to the reader thread
[Wednesday 15 September 2010] [12:01:13] <sustrik> the only point where the two interact is when reader has no more data to read
[Wednesday 15 September 2010] [12:01:21] <sustrik> then it does CAS
[Wednesday 15 September 2010] [12:01:33] <sustrik> to get the writer’s portion of the list
[Wednesday 15 September 2010] [12:02:01] <Samy> So, basically, you batch the operations?
[Wednesday 15 September 2010] [12:02:06] <sustrik> exactly
[Wednesday 15 September 2010] [12:02:09] <sustrik> that’s the key
[Wednesday 15 September 2010] [12:02:14] <Samy> So, writer does enqueue operations to the queue.
[Wednesday 15 September 2010] [12:02:20] <sustrik> ack
[Wednesday 15 September 2010] [12:02:21] <Samy> The reader occassionally does a batch dequeue.
[Wednesday 15 September 2010] [12:02:24] <sustrik> ack
[Wednesday 15 September 2010] [12:02:27] <Samy> Ok.
[Wednesday 15 September 2010] [12:02:48] <Samy> sustrik, why do you use CAS for the batch dequeue? That can be implemented using an xchg, making the batch dequeue a wait-free operation.
[Wednesday 15 September 2010] [12:02:48] <sustrik> it’s pretty effective just because of the batching
[Wednesday 15 September 2010] [12:03:24] <sustrik> hm, i haven’t seen the code for a long time
[Wednesday 15 September 2010] [12:03:27] <sustrik> let me have a look
[Wednesday 15 September 2010] [12:04:02] <Samy> It looks like you do a single CAS.
[Wednesday 15 September 2010] [12:04:12] <Samy> Line 137, check_read (I assume?)
[Wednesday 15 September 2010] [12:04:56] * Samy looks at yqueue
[Wednesday 15 September 2010] [12:05:01] <sustrik> “If there are no
[Wednesday 15 September 2010] [12:05:01] <sustrik> // items to prefetch, set c to NULL”
[Wednesday 15 September 2010] [12:05:09] <sustrik> Samy: yes
[Wednesday 15 September 2010] [12:05:27] <sustrik> the cas is used to communicate back to the writer
[Wednesday 15 September 2010] [12:05:28] <Samy> You can implement this without any lock-free operations.
[Wednesday 15 September 2010] [12:05:43] * sustrik is listening
[Wednesday 15 September 2010] [12:06:32] <sustrik> you mean without atomic ops?
[Wednesday 15 September 2010] [12:06:41] <sustrik> or bus locking?
[Wednesday 15 September 2010] [12:07:14] <Samy> Without explicit atomic operations.
[Wednesday 15 September 2010] [12:07:40] <Samy> sustrik, the idea is to share a stub node for both tail and head of the queue.
[Wednesday 15 September 2010] [12:08:16] <sustrik> can you explain in more detail?
[Wednesday 15 September 2010] [12:08:17] <Samy> sustrik, writer updates tail, always updating the next pointer, and reader always assume first node is stub entry.
[Wednesday 15 September 2010] [12:08:31] <Samy> sustrik, I can show you source-code.
[Wednesday 15 September 2010] [12:08:39] <Samy> Let me see if I find it, hold on.
[Wednesday 15 September 2010] [12:09:02] <sustrik> ok
[Wednesday 15 September 2010] [12:09:35] <sustrik> how does the communication between two cpu cores happens then?
[Wednesday 15 September 2010] [12:09:41] <sustrik> cache coherency algos?
[Wednesday 15 September 2010] [12:11:21] <Samy> sustrik, yes.
[Wednesday 15 September 2010] [12:11:35] <Samy> sustrik, the point is, the only time we need to really share state is if the queue is empty.
[Wednesday 15 September 2010] [12:11:43] <sustrik> ack
[Wednesday 15 September 2010] [12:11:50] <Samy> sustrik, we need a way to detect if the queue is empty atomically and update both head and tail atomically.
[Wednesday 15 September 2010] [12:11:54] <Samy> sustrik, by sharing a stub node, this is possible.
[Wednesday 15 September 2010] [12:12:23] <sustrik> i think something like that is done in ypipe
[Wednesday 15 September 2010] [12:12:33] <sustrik> there’s an emty item in the list
[Wednesday 15 September 2010] [12:12:40] <sustrik> that servers as a placeholder
[Wednesday 15 September 2010] [12:12:50] <sustrik> that’s what you mean by stub, right?
[Wednesday 15 September 2010] [12:12:52] <Samy> Yes.
[Wednesday 15 September 2010] [12:13:20] <Samy> This isn’t fenced correctly, but that should be fine on IA32.
[Wednesday 15 September 2010] [12:14:08] <sustrik> i thought that IA requires you to fence explicitely
[Wednesday 15 September 2010] [12:14:23] <sustrik> ah, you mean x85
[Wednesday 15 September 2010] [12:14:25] <sustrik> 86
[Wednesday 15 September 2010] [12:14:57] <Samy> Yes.
[Wednesday 15 September 2010] [12:15:25] <Samy> The fences may be sufficient, I just haven’t verified it on non-IA32 yet. :-)
[Wednesday 15 September 2010] [12:15:25] <sustrik> ok, there’s one more issue there
[Wednesday 15 September 2010] [12:15:48] <sustrik> when reader finds out that there are no more items to read
[Wednesday 15 September 2010] [12:15:53] <sustrik> it goes asleep
[Wednesday 15 September 2010] [12:16:07] <sustrik> it’s writers responsibity to wake it up
[Wednesday 15 September 2010] [12:16:29] <sustrik> so the writer has to be informed about the fact that reader tried to get more items and failed
[Wednesday 15 September 2010] [12:16:41] <Samy> sustrik, futex(2) works well.
[Wednesday 15 September 2010] [12:16:49] <sustrik> yes, it’s similar
[Wednesday 15 September 2010] [12:17:10] <sustrik> but keep in mind that this is a multi-platform app
[Wednesday 15 September 2010] [12:17:47] <Samy> Things sort of suck without futex. :-(
[Wednesday 15 September 2010] [12:18:01] <Samy> (or a similar mechanism, at least in ring-3)
[Wednesday 15 September 2010] [12:18:08] <sustrik> that’s how it is :|
[Wednesday 15 September 2010] [12:18:28] <Samy> But you could abstract a generic CV layer that uses futex directly if available.
[Wednesday 15 September 2010] [12:18:40] <Samy> sustrik, where is this wake-up mechanism implemented?
[Wednesday 15 September 2010] [12:18:59] <sustrik> out of the class, it’s a dumb socket pair
[Wednesday 15 September 2010] [12:19:09] <sustrik> i had a futex implementation once
[Wednesday 15 September 2010] [12:19:15] <Samy> What happened?
[Wednesday 15 September 2010] [12:19:22] <sustrik> but it was pain in the ass to make it work everywhere
[Wednesday 15 September 2010] [12:19:28] <Samy> I see.
[Wednesday 15 September 2010] [12:19:32] <sustrik> some linux kernels pretend to have futexes
[Wednesday 15 September 2010] [12:19:46] <sustrik> but astucally return ENOTSUP when you try to use them
[Wednesday 15 September 2010] [12:19:47] <sustrik> and alike
[Wednesday 15 September 2010] [12:19:57] <Samy> I see.
[Wednesday 15 September 2010] [12:20:14] <sustrik> anyway, the wake up mechanism is irrelevant for this discussion
[Wednesday 15 September 2010] [12:20:31] <Samy> Mostly, yes.
[Wednesday 15 September 2010] [12:20:36] <sustrik> what’s relevant is the writer has to be notified that reader is sleeping
[Wednesday 15 September 2010] [12:20:46] <sustrik> that’s what the second cas is for
[Wednesday 15 September 2010] [12:20:55] <sustrik> in flush function
[Wednesday 15 September 2010] [12:21:17] <Samy> I don’t understand.
[Wednesday 15 September 2010] [12:21:30] <Samy> Why not simply use plain loads and stores if this is single reader and consumer?
[Wednesday 15 September 2010] [12:22:06] <sustrik> because one bit of information is passed the other way round
[Wednesday 15 September 2010] [12:22:10] <sustrik> from reader to writer
[Wednesday 15 September 2010] [12:22:19] <sustrik> “i am sleeping”
[Wednesday 15 September 2010] [12:22:28] <sustrik> that’s when c is set to NULL
[Wednesday 15 September 2010] [12:22:50] <sustrik> maybe it can be done without atomic ops, i am not an expert
[Wednesday 15 September 2010] [12:23:20] <sustrik> however, if it was a normal locking code, this would be a place where races can occur
[Wednesday 15 September 2010] [12:24:12] <Samy> Ok.
[Wednesday 15 September 2010] [12:24:40] <sustrik> so, reader tries to get the latest batch of the items
[Wednesday 15 September 2010] [12:24:51] <sustrik> and if there are none, it sets c to NULL
[Wednesday 15 September 2010] [12:25:29] <sustrik> in flush function, writer adds new batch of items and if it finds out that c was NULL previously
[Wednesday 15 September 2010] [12:25:35] <sustrik> it knows the reader is sleeping
[Wednesday 15 September 2010] [12:25:42] <sustrik> and that it should wake it up
[Wednesday 15 September 2010] [12:26:11] <sustrik> my feeling is that the operations on c have to be atomic
[Wednesday 15 September 2010] [12:26:18] <sustrik> not 100% sure though
[Wednesday 15 September 2010] [12:30:42] <Samy> read_asleep = false; loop { old = xchg(top, NULL); if old == null then reader_asleep = true; sleep(); }
[Wednesday 15 September 2010] [12:31:25] <Samy> Sorry, read_asleep is in loop.
[Wednesday 15 September 2010] [12:32:40] <sustrik> that’s the writer side?
[Wednesday 15 September 2010] [12:33:50] <Samy> Reader.
[Wednesday 15 September 2010] [12:34:09] <Samy> There is still a risk of spurious wake-up, but that isn’t a big deal.
[Wednesday 15 September 2010] [12:34:17] <Samy> If it is, then you basically implement a barrier.
[Wednesday 15 September 2010] [12:34:55] <Samy> You have a reader and a writer shared variable. Writer signals reader once, only if reader has signaled writer since last wake-up.
[Wednesday 15 September 2010] [12:35:55] <sustrik> yes, that’s the goal
[Wednesday 15 September 2010] [12:36:03] <Samy> And the algorithm.
[Wednesday 15 September 2010] [12:36:15] * sustrik is a dummy
[Wednesday 15 September 2010] [12:36:21] <Samy> Me too.
[Wednesday 15 September 2010] [12:36:28] <sustrik> can you give a pseudo-code for writer side as well?
[Wednesday 15 September 2010] [12:38:19] <sustrik> in the code above reader_asleep is another shared variable, right?
[Wednesday 15 September 2010] [12:39:32] <pieterh> sustrik: I was trying to say something about options but got interrupted by stuff...
[Wednesday 15 September 2010] [12:40:02] <pieterh> API should IMO reject an identity starting with zero byte
[Wednesday 15 September 2010] [12:40:19] <sustrik> now you are likely to get interrupted by brain-damaging lock-free algo discussion :)
[Wednesday 15 September 2010] [12:40:33] <pieterh> :-/
[Wednesday 15 September 2010] [12:40:34] <sustrik> yes, that would be nice
[Wednesday 15 September 2010] [12:41:04] <Samy> reader: loop { r = false; store_fence; w = true; o = xchg(top, NULL); if old == null then { r = true; sleep(); } }
[Wednesday 15 September 2010] [12:41:05] <Samy> writer: add(); c_w = w; load_fence; c_r = r; if c_r == true and c_w == true; then { c_w = false; signal_reader; }
[Wednesday 15 September 2010] [12:41:24] <sustrik> pieter: fancy sumbitting a patch?
[Wednesday 15 September 2010] [12:41:49] <pieterh> sustrik: I’m pulling master as we speak, yes I’ll make you a patch
[Wednesday 15 September 2010] [12:42:45] <sustrik> thanks
[Wednesday 15 September 2010] [12:42:56] <sustrik> Samy: sorry, having 3 discussions in parallel
[Wednesday 15 September 2010] [12:43:34] <Samy> Hey, no problem. :-)
[Wednesday 15 September 2010] [12:43:39] <Samy> I’ll be back in some minutes.
[Wednesday 15 September 2010] [12:43:53] <sustrik> sure, checking your code right now
[Wednesday 15 September 2010] [12:46:14] <sustrik> what’s top?
[Wednesday 15 September 2010] [12:46:50] <sustrik> which variable belong to whom?
[Wednesday 15 September 2010] [12:46:57] <sustrik> both r and w seem to be shared
[Wednesday 15 September 2010] [12:48:23] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 15 September 2010] [12:49:34] <pieterh> sustrik: np. options.cpp already checks that... :-)
[Wednesday 15 September 2010] [12:49:43] <sustrik> good
[Wednesday 15 September 2010] [12:51:56] <Samy> sustrik, yes, they are shared.
[Wednesday 15 September 2010] [12:52:46] <Samy> sustrik, that statement is whatever condition you use to check if the queue is empty.
[Wednesday 15 September 2010] [12:53:17] <sustrik> this one: o = xchg(top, NULL); ?
[Wednesday 15 September 2010] [12:53:25] <Samy> That and next.
[Wednesday 15 September 2010] [12:53:34] <Samy> o == null
[Wednesday 15 September 2010] [12:53:35] <Samy> (not old)
[Wednesday 15 September 2010] [12:54:11] <sustrik> do c_r and c_w are local to writer
[Wednesday 15 September 2010] [12:54:17] <Samy> Yes.
[Wednesday 15 September 2010] [12:54:32] * sustrik is trying to make sense of it
[Wednesday 15 September 2010] [12:54:58] <Samy> The reason we have 2 variables is that writer will signal the reader only once.
[Wednesday 15 September 2010] [12:55:40] <Samy> The reader will then wake-up, and if the queue is empty it will indicate that it is asleep and it is fine for the writer to send another signal.
[Wednesday 15 September 2010] [12:56:22] <Samy> sustrik, if you treat it like a stack, it will make more sense.
[Wednesday 15 September 2010] [12:56:50] <sustrik> Samy: stack of what?
[Wednesday 15 September 2010] [12:56:54] <Samy> sustrik, for a FIFO, you can use another technique.
[Wednesday 15 September 2010] [12:57:04] <Samy> sustrik, of objects.
[Wednesday 15 September 2010] [12:57:11] <sustrik> that’s the case in 0MQ
[Wednesday 15 September 2010] [12:57:20] <Samy> sustrik, this is in reference of xchg(top, ...) :)
[Wednesday 15 September 2010] [12:57:32] <Samy> sustrik, what’s of relevance to you is the notion of the r and w variables, that’s all.
[Wednesday 15 September 2010] [12:58:03] <sustrik> r = “i am sleeeping”
[Wednesday 15 September 2010] [12:58:14] <sustrik> what abour w?
[Wednesday 15 September 2010] [12:58:19] <sustrik> about*
[Wednesday 15 September 2010] [12:58:39] <Samy> w = “reader has woken-up since the last signal”
[Wednesday 15 September 2010] [12:59:20] <sustrik> soemging is missing
[Wednesday 15 September 2010] [12:59:25] <sustrik> w is never set to false
[Wednesday 15 September 2010] [12:59:34] <Samy> Oh, sorry.
[Wednesday 15 September 2010] [12:59:44] <Samy> { c_w = false; signal_reader; } should be { w = false; signal_reader; }
[Wednesday 15 September 2010] [13:00:14] <sustrik> lk,now it makes sense
[Wednesday 15 September 2010] [13:00:56] <sustrik> now, what about the fences
[Wednesday 15 September 2010] [13:01:29] <sustrik> does it sync the data in the queue as well?
[Wednesday 15 September 2010] [13:02:03] <Samy> You can figure that out.
[Wednesday 15 September 2010] [13:02:22] <Samy> That is mainly meant for those flags.
[Wednesday 15 September 2010] [13:02:29] Quit AndrewBC has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [13:02:56] <Samy> The idea, reader should always appear to be set to false before writer is set to true (it must indicate it has woken up before it indicates it is fine to send another signal).
[Wednesday 15 September 2010] [13:03:42] <sustrik> got it
[Wednesday 15 September 2010] [13:04:14] <sustrik> that part ensures that everything happens in nice step-lock fashion
[Wednesday 15 September 2010] [13:04:19] <Samy> Right.
[Wednesday 15 September 2010] [13:04:42] <sustrik> now few technical questions...
[Wednesday 15 September 2010] [13:04:49] <Samy> I’m no expert, but ok.
[Wednesday 15 September 2010] [13:05:02] <sustrik> what does lock;xchg() actually means
[Wednesday 15 September 2010] [13:05:11] <sustrik> is there an implied barrier?
[Wednesday 15 September 2010] [13:05:29] <Samy> On IA32, atomic operations have a total order across all processors.
[Wednesday 15 September 2010] [13:05:46] <Samy> There is an implicit pipeline flush.
[Wednesday 15 September 2010] [13:06:11] <Samy> There is an implied barrier.
[Wednesday 15 September 2010] [13:06:24] <Samy> There is also no need for the lock prefix on xchg.
[Wednesday 15 September 2010] [13:06:41] <Samy> xchg is guaranteed to be atomic (which is what also makes it expensive to use, if you don’t need it).
[Wednesday 15 September 2010] [13:06:42] <sustrik> in that case there’s no global ordering, right?
[Wednesday 15 September 2010] [13:06:52] <Samy> Usually, yes.
[Wednesday 15 September 2010] [13:07:10] <Samy> add is not guaranteed to be atomic, lock add is.
[Wednesday 15 September 2010] [13:07:16] <Samy> xchg is always atomic.
[Wednesday 15 September 2010] [13:07:21] <sustrik> oh my
[Wednesday 15 September 2010] [13:07:41] <Samy> It’s expensive compared to simple loads and stores, that’s for sure.
[Wednesday 15 September 2010] [13:07:41] <sustrik> so there’s an implied barrier on xchg, right?
[Wednesday 15 September 2010] [13:07:45] <Samy> Yes.
[Wednesday 15 September 2010] [13:07:58] <sustrik> then, in your code we have 2 barriers
[Wednesday 15 September 2010] [13:08:03] <Samy> For more information, you can see Volume 3 of the Intel Architecture Manuals, Chapter 8.
[Wednesday 15 September 2010] [13:08:37] <sustrik> isn’t it better to use single cas with a barrier then?
[Wednesday 15 September 2010] [13:08:38] <Samy> sustrik, you don’t need any of those barriers for IA32. Stores are always seen in order.
[Wednesday 15 September 2010] [13:09:19] <Samy> sustrik, CAS is expensive.
[Wednesday 15 September 2010] [13:09:31] <Samy> sustrik, it’s best to avoid the lock prefix completely if possible.
[Wednesday 15 September 2010] [13:09:37] <sustrik> ack
[Wednesday 15 September 2010] [13:09:41] <Samy> sustrik, atomic loads and stores do this.
[Wednesday 15 September 2010] [13:10:08] <sustrik> atomic load & stores == have global ordering ?
[Wednesday 15 September 2010] [13:10:12] <Samy> sustrik, in theory, CAS can provide infinite consensus. If you don’t need infinite consensus, you might not need CAS at all.
[Wednesday 15 September 2010] [13:10:50] <sustrik> anyway, i have to get my head around it
[Wednesday 15 September 2010] [13:11:00] <sustrik> are you around somewhere?
[Wednesday 15 September 2010] [13:11:01] <Samy> Processor ordering.
[Wednesday 15 September 2010] [13:11:10] <sustrik> email, or so?
[Wednesday 15 September 2010] [13:11:29] <Samy> sustrik, I’d love your feedback on this library I’m working on sometime.
[Wednesday 15 September 2010] [13:11:36] <Samy> sustrik, what architectures does ZeroMQ support?
[Wednesday 15 September 2010] [13:12:26] <sustrik> we’ve tested on x86, itanium, sparc, ppc
[Wednesday 15 September 2010] [13:12:28] <sustrik> arm
[Wednesday 15 September 2010] [13:12:29] <Samy> sustrik, if you see section 8.2.2 in Volume 3 of the IA32 manuals, you’ll get a nice break-down of the memory ordering.
[Wednesday 15 September 2010] [13:12:58] <Samy> sustrik, does the ZeroMQ project have regular access to such boxes?
[Wednesday 15 September 2010] [13:13:21] <sustrik> we have an itanium and sparc box
[Wednesday 15 September 2010] [13:13:28] <sustrik> no ppc
[Wednesday 15 September 2010] [13:13:28] <Samy> Itanium, nice. :-)
[Wednesday 15 September 2010] [13:13:38] <Samy> My PPC box died (Mac Mini) recently. Need to fix it.
[Wednesday 15 September 2010] [13:14:01] <Samy> I sold my only decent SPARCv9 box, at work our SPARCv9 machines are tied to our build cluster.
[Wednesday 15 September 2010] [13:14:21] <Samy> sustrik, if I could get accounts on those, I can port my library’s atomic interface for them to support those.
[Wednesday 15 September 2010] [13:14:31] <Samy> sustrik, might be useful, at least as a reference implementation of some structures.
[Wednesday 15 September 2010] [13:15:09] <Samy> sustrik, ;]
[Wednesday 15 September 2010] [13:15:46] <sustrik> our sparc box is extremely old and slow :)
[Wednesday 15 September 2010] [13:15:54] <Samy> Itanium is fine too.
[Wednesday 15 September 2010] [13:16:07] <Samy> Well, Itanium is what I really want access to.
[Wednesday 15 September 2010] [13:16:10] <sustrik> but mato mumbled something about bringing some 14-core sparc
[Wednesday 15 September 2010] [13:16:16] <sustrik> ah
[Wednesday 15 September 2010] [13:16:36] <sustrik> i have to ask mato, i think there’s some applications running on it
[Wednesday 15 September 2010] [13:16:42] <keffo> sandy bridge ftw!
[Wednesday 15 September 2010] [13:16:48] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [13:17:20] <Samy> I look forward to future discussions. I would like to show you the work I’ve done so far for feedback.
[Wednesday 15 September 2010] [13:17:34] <Samy> It is C, not C++, however.
[Wednesday 15 September 2010] [13:17:41] <sustrik> Samy, i can have a look but as you see i am not an expert :)
[Wednesday 15 September 2010] [13:18:13] <Samy> Well, that’s fine. The idea is, what would it take to make you use this? As a 3rd party looking to integrate this into their product, what issues do you have with the interface?
[Wednesday 15 September 2010] [13:18:28] <Samy> etc
[Wednesday 15 September 2010] [13:18:35] <sustrik> what’s the link to the lib?
[Wednesday 15 September 2010] [13:18:43] <kleppari> we’re ditching a couple of sun t1000 boxes at work
[Wednesday 15 September 2010] [13:18:48] <Samy> No link. Tarball, sustrik.
[Wednesday 15 September 2010] [13:18:54] <Samy> When we have a discussion, I can share that.
[Wednesday 15 September 2010] [13:18:55] <kleppari> let me see if I can ‘rescue’ on of them
[Wednesday 15 September 2010] [13:19:07] <kleppari> s/on/one/
[Wednesday 15 September 2010] [13:19:19] Quit mrm2m has left this server (Quit: Leaving.).
[Wednesday 15 September 2010] [13:20:11] <Samy> Cool.
[Wednesday 15 September 2010] [13:20:16] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [13:20:19] <Samy> I’m back to work, take care.
[Wednesday 15 September 2010] [13:20:41] <sustrik> you too
[Wednesday 15 September 2010] [13:20:42] <sustrik> bye
[Wednesday 15 September 2010] [13:20:47] <sustrik> kleppari: i don’t think it’s really needed
[Wednesday 15 September 2010] [13:21:15] <sustrik> unless you need it yourself
[Wednesday 15 September 2010] [13:21:41] <kleppari> not really
[Wednesday 15 September 2010] [13:21:47] <kleppari> kind of lost interest in solaris after oracle bought it
[Wednesday 15 September 2010] [13:21:53] <kleppari> err, after oracle bought sun
[Wednesday 15 September 2010] [13:22:00] <kleppari> I think they’ll do a wonderful job killing it
[Wednesday 15 September 2010] [13:22:23] <sustrik> yeah, looks like there will be a lot of sun boxes avialable :)
[Wednesday 15 September 2010] [13:22:42] <kleppari> heh, yeah - red hat will make a fortune
[Wednesday 15 September 2010] [13:23:37] <kleppari> but the t1000 was a good box, I still think 16 concurrent threads of execution per core is impressive
[Wednesday 15 September 2010] [13:24:57] <kleppari> t2000, sorry
[Wednesday 15 September 2010] [13:27:14] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [13:29:59] <pieterh> kleppari: we used to have a t2000 at iMatix when we were making OpenAMQ
[Wednesday 15 September 2010] [13:30:11] <pieterh> it was a pretty impressive box, made a noise like an aircraft taking off
[Wednesday 15 September 2010] [13:30:30] <pieterh> and was slower (all 32 cores or whatever) than a 2-core Athlon
[Wednesday 15 September 2010] [13:31:10] Quit ModusPwnens has left this server.
[Wednesday 15 September 2010] [13:31:10] <mato> pieterh: it still might be more useful than the old e4500 i can get for the project
[Wednesday 15 September 2010] [13:31:16] <kleppari> slower at what?
[Wednesday 15 September 2010] [13:31:22] <mato> pieterh: 0mq is a very different codebase from OpenAMQ
[Wednesday 15 September 2010] [13:32:01] <pieterh> mato: perhaps, yes
[Wednesday 15 September 2010] [13:32:12] <mato> kleppari: if you do get a chance to rescue one i think it’d be more interesting for development of the lockfree algorithms in 0mq than what i’ve been offered
[Wednesday 15 September 2010] [13:32:14] <pieterh> kleppari: at raw I/O afaics
[Wednesday 15 September 2010] [13:32:24] <pieterh> but the main difference was probably Linux vs. Solaris
[Wednesday 15 September 2010] [13:32:25] <mato> kleppari: which is an old E4500 (fully spec’ed with 14 CPUs)
[Wednesday 15 September 2010] [13:32:34] <kleppari> I found that these boxes perform pretty well with a huge runqueue
[Wednesday 15 September 2010] [13:32:42] <kleppari> mato: let me see what I can do
[Wednesday 15 September 2010] [13:32:50] <pieterh> they were designed for web services, indeed
[Wednesday 15 September 2010] [13:32:56] <kleppari> mato: can’t promise anything, though, but I’ll try
[Wednesday 15 September 2010] [13:32:58] Quit xla has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [13:33:10] <mato> kleppari: no hurry; also, where are you located? it’s not worth shipping that stuff from outside of the EU, too expensive
[Wednesday 15 September 2010] [13:33:18] <kleppari> iceland
[Wednesday 15 September 2010] [13:33:22] <mato> lol :)
[Wednesday 15 September 2010] [13:33:29] <kleppari> so the shipping might kill the deal :P
[Wednesday 15 September 2010] [13:33:42] <mato> kleppari: precisely, unless you have friends at smyril line :-)
[Wednesday 15 September 2010] [13:33:54] <mato> “chuck it down with them bananas” :-)
[Wednesday 15 September 2010] [13:34:12] <kleppari> heheh
[Wednesday 15 September 2010] [13:35:50] <kleppari> I don’t think we’d be shipping bananas out of the country, but fair point :P
[Wednesday 15 September 2010] [13:36:15] <pieterh> hey... ebay is offering free shipping or something...
[Wednesday 15 September 2010] [13:36:54] <mato> kleppari: thanks for the offer in any case
[Wednesday 15 September 2010] [13:36:58] * mato is off to 0beer :-)
[Wednesday 15 September 2010] [13:37:05] <pieterh> 0beer0clock
[Wednesday 15 September 2010] [13:37:08] <mato> beer
[Wednesday 15 September 2010] [13:37:12] <pieterh> zbeer
[Wednesday 15 September 2010] [13:37:13] <mato> cyl
[Wednesday 15 September 2010] [13:37:18] <pieterh> cyrsn
[Wednesday 15 September 2010] [13:38:10] * pieterh suspects mato has a macro that types
[Wednesday 15 September 2010] [13:38:22] <mato> !
[Wednesday 15 September 2010] [13:38:56] <pieterh> :-) I knew it...
[Wednesday 15 September 2010] [13:39:08] <pieterh> oh... hang on...
[Wednesday 15 September 2010] [13:39:20] Quit AndrewBC has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [13:39:45] * pieterh wonders how to config xmodmap to do this...
[Wednesday 15 September 2010] [13:43:42] <kleppari> alt gr + o ?
[Wednesday 15 September 2010] [13:43:45] <kleppari> works on the .is layout
[Wednesday 15 September 2010] [13:44:07] <guido_g> and on 105 key .de (qwertz)
[Wednesday 15 September 2010] [13:44:39] <guido_g> and I just saw that ch. 3 is back...
[Wednesday 15 September 2010] [13:44:45] <pieterh> yeah, I fixed it
[Wednesday 15 September 2010] [13:45:03] * guido_g is reading
[Wednesday 15 September 2010] [13:45:14] Quit pieterh has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [13:46:48] <pieterh> cyal, i’m off for zbeer
[Wednesday 15 September 2010] [13:47:19] <pieterh> guido_g: there’s also a new problem solver in Ch1
[Wednesday 15 September 2010] [13:47:35] <guido_g> oh...
[Wednesday 15 September 2010] [13:50:32] <guido_g> nice one
[Wednesday 15 September 2010] [13:50:54] <guido_g> but can you do it in uml?
[Wednesday 15 September 2010] [13:51:01] <guido_g> ducks for cover
[Wednesday 15 September 2010] [13:52:53] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [13:53:25] <guido_g> more serious thing: smaller version that fits on one page so it can be printed easily
[Wednesday 15 September 2010] [14:04:28] <keffo> sustrik, What scenario could trigger a deadlock? (in single threaded proc.)
[Wednesday 15 September 2010] [14:59:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [15:09:33] <Samy> sustrik, ping?
[Wednesday 15 September 2010] [15:27:31] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [15:27:41] <ModusPwnens> Does zeromq have a lot of start up overhead of initialization?
[Wednesday 15 September 2010] [15:27:46] <ModusPwnens> or initialization*
[Wednesday 15 September 2010] [15:36:47] <ModusPwnens> the only reason I ask is because i keep getting strange results when I benchmark where the throughput is low intially, but then gradually increases
[Wednesday 15 September 2010] [15:47:34] Quit g5q has left this server (Quit: Bye~).
[Wednesday 15 September 2010] [15:51:53] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [15:54:03] Quit keffo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [16:19:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 15 September 2010] [16:36:23] Quit zirpu has left this server (Quit: kernel upgrade.).
[Wednesday 15 September 2010] [16:44:47] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [18:45:35] Quit sbahra has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [19:27:16] Quit danielqo has left this server (Quit: danielqo).
[Wednesday 15 September 2010] [19:56:19] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [20:24:19] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [21:17:56] Quit carrus85 has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [21:25:08] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [21:33:25] Quit andrewvc has left this server (Quit: andrewvc).
[Wednesday 15 September 2010] [21:39:21] Quit danielqo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [21:42:30] Quit danielqo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [21:48:19] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [21:52:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 15 September 2010] [22:17:24] <lestrrat> is there some sort of debug flag to enable tracing or some such for zeromq?
[Wednesday 15 September 2010] [22:17:46] <lestrrat> I’m having problems, and want to know what’s going on in the zeromq side...
[Wednesday 15 September 2010] [22:56:14] <dermoth> lestrrat, maybe strace would help, but it might be a bit low-level depending on what you need to figure out
[Wednesday 15 September 2010] [22:57:28] <dermoth> lestrrat, fwiw if you’re using short-lived processes then some queues have no way to guarantee all messages have been delivered before shutdown; when you call zmq_send data is only bufered in a local queue to be sent to the remote end
[Wednesday 15 September 2010] [23:03:52] <lestrrat> thanks, gonna have to RFE a TRACE variable or something ;)
[Thursday 16 September 2010] [00:09:44] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [01:06:23] Quit carrus85 has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [01:28:04] Quit guido_g has left this server (Quit: Boom...).
[Thursday 16 September 2010] [01:35:08] Quit carrus85 has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [01:36:14] Quit carrus85 has left this server (Client Quit).
[Thursday 16 September 2010] [01:52:22] Quit keffo has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [01:58:44] Quit
AndrewBC_ has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [02:03:48] Quit AndrewBC has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [03:02:56] Quit kleppari has left this server (Read error: Operation timed out).
[Thursday 16 September 2010] [03:12:21] <dermoth> I was trying out the php-zmq module today, and it appears some tests die with an assertion failed with –enable-pgm in the zeromq library
[Thursday 16 September 2010] [03:12:43] <dermoth> not that I need it though...
[Thursday 16 September 2010] [03:14:05] <dermoth> the error is: Assertion failed: !pgm_supported () (zmq.cpp:240)
[Thursday 16 September 2010] [03:17:20] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [03:33:05] Quit kenkeiter has left this server (Quit: kenkeiter).
[Thursday 16 September 2010] [03:34:16] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [03:47:30] <sustrik> dermoth: don’t ope 2 contexts
[Thursday 16 September 2010] [03:47:33] <sustrik> open
[Thursday 16 September 2010] [04:03:48] Nick Nickname already in use. Trying
travlr_.
[Thursday 16 September 2010] [04:03:49] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 16 September 2010] [04:03:50] CTCP Received Version request from frigg.
[Thursday 16 September 2010] [04:03:51] Topic The channel topic is “Welcome!”.
[Thursday 16 September 2010] [04:03:53] Quit travlr has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [04:03:57] Mode Channel modes: no messages from outside, topic protection
[Thursday 16 September 2010] [04:03:57] Created This channel was created on 2010-02-05 10:44.
[Thursday 16 September 2010] [04:09:59] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Thursday 16 September 2010] [04:22:28] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 16 September 2010] [04:23:19] Quit xla has left this server (Client Quit).
[Thursday 16 September 2010] [04:28:49] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 16 September 2010] [04:32:25] Nick Nickname already in use. Trying
travlr__.
[Thursday 16 September 2010] [04:32:26] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 16 September 2010] [04:32:27] CTCP Received Version request from frigg.
[Thursday 16 September 2010] [04:32:28] Topic The channel topic is “Welcome!”.
[Thursday 16 September 2010] [04:32:34] Mode Channel modes: no messages from outside, topic protection
[Thursday 16 September 2010] [04:32:34] Created This channel was created on 2010-02-05 10:44.
[Thursday 16 September 2010] [04:32:46] Quit
travlr_ has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [04:58:19] Quit xrfang has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [05:26:25] <lestrrat> cool, it wasn’t just me who got bit with multiple contexts + pgm ;)
[Thursday 16 September 2010] [06:05:00] <sustrik> lestrrat: steven mc coy have already started migrating 0mq to new version of openpgm that should have the problem solved
[Thursday 16 September 2010] [06:05:19] <sustrik> but it’s work in progress, no guarantees
[Thursday 16 September 2010] [07:18:29] Quit jsimmons has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [08:03:25] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [09:05:38] <psino> in zmq 2.1, zmq_close wont discard that is queued for sending – is anything similar going to be implemented for data that has arrived, and are pending recv-calls?
[Thursday 16 September 2010] [09:21:00] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [09:30:00] Quit guido_g has left this server (Quit: Boom...).
[Thursday 16 September 2010] [09:44:05] <joyenlee> Hi!
[Thursday 16 September 2010] [09:44:52] Quit joyenlee has left this server (Quit: Page closed).
[Thursday 16 September 2010] [09:49:06] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 16 September 2010] [10:00:22] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [10:02:55] Quit iFire has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [10:30:31] Quit mato has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [10:35:09] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [10:37:42] Quit grahamalot has left this server (Read error: Operation timed out).
[Thursday 16 September 2010] [11:33:31] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [11:35:38] Quit psino has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [11:56:30] Quit psino has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [12:45:29] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [13:48:38] Quit AndrewBC has left this server (Ping timeout: 264 seconds).
[Thursday 16 September 2010] [14:02:55] Quit xla has left this server (Quit: leaving).
[Thursday 16 September 2010] [14:14:38] Quit psino has left this server (Quit: psino).
[Thursday 16 September 2010] [14:17:11] Quit keffo has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [14:17:17] Quit ModusPwnens has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [14:32:59] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [14:45:08] Quit AndrewBC has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [15:07:01] Join yrashk has joined this channel (
~yrashk@70.28.245.3).
[Thursday 16 September 2010] [15:09:10] Quit fredix has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [15:09:22] Quit omarkj has left this server (Quit: omarkj).
[Thursday 16 September 2010] [15:14:22] Join fredix has joined this channel (
~fredix@2a01:e35:2f38:d790:218:2dff:fe00:173).
[Thursday 16 September 2010] [15:56:08] Quit yrashk has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [17:15:50] <dermoth> sustrik, i didn’t write the test... but why would it fail? What is multiple applications use different context on the same server? do you think the contect should be abstracted in the PHP module?
[Thursday 16 September 2010] [17:43:05] Quit g5q has left this server (Quit: Bye~).
[Thursday 16 September 2010] [18:12:59] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Thursday 16 September 2010] [18:13:23] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [18:36:54] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [18:42:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [18:46:48] Quit keffo has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [19:13:13] Quit andrewvc has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [19:21:52] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [20:11:41] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 16 September 2010] [20:15:04] Quit jhawk28 has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [20:45:07] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Thursday 16 September 2010] [20:46:51] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [20:57:43] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [21:01:31] Quit plq has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [21:23:48] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [21:24:04] Quit jhawk28 has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [21:31:14] Quit erickt has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [21:41:14] <
mattp_> Hi. I’m trying to build using Cygwin. I’m getting me an error message telling me to install e2fsprog even though I’ve already got it.
[Thursday 16 September 2010] [21:41:18] <
mattp_> Any suggestions?
[Thursday 16 September 2010] [21:42:22] <
mattp_> (not sure how that “me” got into my third sentence. :) )
[Thursday 16 September 2010] [21:49:23] <
mattp_> crickets...
[Thursday 16 September 2010] [21:50:38] Quit
mattp_ has left this server (Quit: Using leafChat 2).
[Thursday 16 September 2010] [21:51:44] Quit
mattp_ has left this server (Client Quit).
[Thursday 16 September 2010] [21:52:32] <mpassell> thought I’d try through a different client. Anyone out there?
[Thursday 16 September 2010] [22:01:10] Part mpassell has left this channel.
[Thursday 16 September 2010] [22:12:57] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [23:28:16] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Thursday 16 September 2010] [23:51:02] <chowmeined> can one multiplex different message patterns on a single socket?
[Friday 17 September 2010] [00:18:42] <andrewvc> cremes, by chance are you around?
[Friday 17 September 2010] [00:19:25] Part snerd has left this channel.
[Friday 17 September 2010] [00:33:45] <dermoth> iirc there used to be device config example, looks like they’re gone. why?
[Friday 17 September 2010] [00:33:56] <dermoth> king of easier than reading the code :)
[Friday 17 September 2010] [00:38:41] <guido_g> maybe thats what you want
[Friday 17 September 2010] [00:40:27] <dermoth> guido_g, thanks, looks nice. Actually for now i’m just using the stock devices that come with zeromq. The config quite basic, in xml. I was justt wondering where they gone
[Friday 17 September 2010] [00:40:36] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [00:41:11] <guido_g> afair there going to be moved out of the core distribution
[Friday 17 September 2010] [00:42:44] <dermoth> ok... well I kind of like them for the most basic usage... I could replace the config with command-line arguments though, imho it would be much easier
[Friday 17 September 2010] [00:43:05] <guido_g> dermoth: btw, please send your patch to the ml with a statement that it is licensed under mit/x11, othwerwise it can’t be accepted
[Friday 17 September 2010] [00:43:33] <dermoth> really?
[Friday 17 September 2010] [00:44:25] <guido_g> it’s the procedure
[Friday 17 September 2010] [00:44:50] <guido_g> has been discussed here and on the ml
[Friday 17 September 2010] [00:46:13] <guido_g> dermoth: easiest way is to use format-patch and append that to the mal w/ the license statement
[Friday 17 September 2010] [00:46:15] <dermoth> then how about I sen you this?
[Friday 17 September 2010] [00:46:19] <dermoth> perl -pi -e ‘s/ZMQ_DOWNSTREAM/ZMQ_PUSH/; s/ZMQ_UPSTREAM/ZMQ_PULL/;’ devices/zmq_streamer/zmq_streamer.cpp
[Friday 17 September 2010] [00:46:20] <guido_g> not me
[Friday 17 September 2010] [00:46:36] <guido_g> dermoth: see git format-patch
[Friday 17 September 2010] [00:46:50] <dermoth> git commit -m’zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL’
[Friday 17 September 2010] [00:47:12] <dermoth> I use git-format-patch... but I’m not even subscrived to the ml
[Friday 17 September 2010] [00:47:29] <guido_g> too bad
[Friday 17 September 2010] [00:47:35] <dermoth> the two commands abot will get you the same result less the credit
[Friday 17 September 2010] [00:48:02] <guido_g> what credit?
[Friday 17 September 2010] [00:48:11] <dermoth> the author in commit message
[Friday 17 September 2010] [00:48:33] <guido_g> ah
[Friday 17 September 2010] [00:48:55] <dermoth> unless you’re authorised to do: git commit -m’zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL’ ‘–author=Thomas Guyot-Sionnest <
thomas@blinkx.com>’
[Friday 17 September 2010] [00:48:56] <guido_g> i’m more concerned about the license statement, w/o it the patch can’t go in
[Friday 17 September 2010] [00:49:40] <dermoth> it’s not a patch it a command that patsh, there shouldn’t be any legal issue. if you run the command you have the copyright for the resulting code ;)
[Friday 17 September 2010] [00:50:13] <guido_g> i think that is clear
[Friday 17 September 2010] [00:50:50] <guido_g> i just stated what’s needed to get a patch in, because a pull-req is a patch
[Friday 17 September 2010] [00:54:01] Quit kenkeiter has left this server (Quit: kenkeiter).
[Friday 17 September 2010] [00:54:07] <dermoth> I understand the legal basis, but I will likely not send a patch that takes me more time to “send” thna write... Imagine trying that with the Linux Kernel!
[Friday 17 September 2010] [00:54:48] <guido_g> as i said, the device apps will be moved out of the core, afaik
[Friday 17 September 2010] [00:55:01] <guido_g> so...
[Friday 17 September 2010] [00:55:12] <dermoth> so I guess there won’t be any trouble getting that in, whatever method you use... on that i’ll get back to my testing :)
[Friday 17 September 2010] [00:55:25] <guido_g> dermoth: wrong
[Friday 17 September 2010] [00:55:35] <guido_g> the license issue is a problem atm
[Friday 17 September 2010] [00:55:41] <guido_g> ask pieter about it
[Friday 17 September 2010] [00:56:17] <guido_g> and go read the ml (via gmane w/o registration)
[Friday 17 September 2010] [00:56:51] <dermoth> what I meant is that it probably take you less time to rename it yourself than using my patch (and I use gmane...)
[Friday 17 September 2010] [00:57:57] <dermoth> i wouldn’t mind taking the time to post on the ml for a real change... and can’T I put it in the commit message too?
[Friday 17 September 2010] [00:58:14] <guido_g> cool, then you can easily find the threads about patches and legal issues
[Friday 17 September 2010] [00:58:43] <guido_g> if it were in the commit message, even better
[Friday 17 September 2010] [00:58:44] <dermoth> except that i’d rather catch some much needed sleep
[Friday 17 September 2010] [00:58:48] <dermoth> ok
[Friday 17 September 2010] [01:04:50] <dermoth> here you go... next time i’ll know :)
[Friday 17 September 2010] [01:12:46] <guido_g> hey, i didn’t invent the laws and such, just repeating the handbook
[Friday 17 September 2010] [01:13:07] <dermoth> Any PECL developers here? IMHO the PHP api should have a cb that is called on persistent socket creation, so we could create the socket and connect in one shot
[Friday 17 September 2010] [01:13:24] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [01:33:13] Quit carrus85 has left this server (Quit: Leaving).
[Friday 17 September 2010] [01:33:57] Join kenkeiter has joined this channel (
~kenkeiter@2002:4c73:685:1234:223:6cff:fe98:be3b).
[Friday 17 September 2010] [01:36:54] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [01:37:00] Quit kenkeiter has left this server (Client Quit).
[Friday 17 September 2010] [01:40:32] <sleeperbot> this driver takes up a ton of CPU
[Friday 17 September 2010] [01:42:32] <sleeperbot> the way that it is implemented =/
[Friday 17 September 2010] [01:48:29] <yrashk> sleeperbot afair there’s none
[Friday 17 September 2010] [01:48:41] <yrashk> but supposedly I will need one soon-ish
[Friday 17 September 2010] [01:48:54] <yrashk> so might invest some time in getting it done properly
[Friday 17 September 2010] [01:51:51] <sleeperbot> i see, thanks for the info
[Friday 17 September 2010] [01:52:55] <yrashk> yw
[Friday 17 September 2010] [01:54:32] Quit keffo has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [02:20:26] Quit sleeperbot has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [02:44:10] Quit jsimmons has left this server (Client Quit).
[Friday 17 September 2010] [02:52:57] Quit xrfang has left this server (Remote host closed the connection).
[Friday 17 September 2010] [03:02:27] <sustrik> psino: how would it work for recv calls?
[Friday 17 September 2010] [03:03:38] <sustrik> dermoth: it’s a problem with OpenPGM, which will be solved soon; what i was saying is that in most cases you have just one context so you never encounter the problem
[Friday 17 September 2010] [03:04:38] <sustrik> chowmeined: no, you can’t
[Friday 17 September 2010] [03:04:59] Quit kleppari has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [03:06:05] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Friday 17 September 2010] [03:09:17] Quit guido_g has left this server (Quit: Boom...).
[Friday 17 September 2010] [03:21:08] Quit fredix has left this server (Read error: Operation timed out).
*** on Fri Sep 17 04:13:00 2010
[Friday 17 September 2010] [04:13:00] Topic The channel topic is “Welcome!”.
[Friday 17 September 2010] [04:13:04] Mode Channel modes: no messages from outside, topic protection
[Friday 17 September 2010] [04:13:04] Created This channel was created on 2010-02-05 10:44.
[Friday 17 September 2010] [04:49:41] <ivan-korobkov> Hi, I use zmq 2.0.9 + epgm + the latest python bindings. Sometimes it crashes with Assertion failed:
*tmpbuf > 0 (zmq_decoder.cpp:60). Does anybody know why it happens? Is it a bug and I should create an issue?
[Friday 17 September 2010] [05:05:08] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 17 September 2010] [05:08:09] <guido_g> try to isolate it in a small test case, so others can verify it
[Friday 17 September 2010] [05:12:03] <ivan-korobkov> OK, I will try, but it happens rather sporadically.
[Friday 17 September 2010] [05:13:15] <guido_g> will be no problem, if we have a program that can run
[Friday 17 September 2010] [05:17:15] Quit xrfang has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [05:31:16] <psino> sustrik: In the case of a pull socket, I was thinking it could disconnect from the push socket and still be able to recv() on the messages the push socket has sent to it
[Friday 17 September 2010] [05:40:43] <sustrik> ivan-korobkov: create an issue
[Friday 17 September 2010] [05:41:15] <sustrik> psino: any idea of how the API should look like?
[Friday 17 September 2010] [05:46:23] <psino> sustrik: from the end users point of view, much like now, except that after calling socket.close(), a poller wouldn’t put the socket in exceptional state before the messages that has been sent to it from the other side has been recv()’ed
[Friday 17 September 2010] [05:49:07] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [05:56:48] Quit mato has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:10:01] Quit ivan-korobkov has left this server (Quit: Leaving).
[Friday 17 September 2010] [06:15:51] Quit guido_g has left this server (Quit: Boom...).
[Friday 17 September 2010] [06:16:50] Quit iFire has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:28:18] Quit
AndrewBC_ has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:28:19] Quit plq has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:32:59] <CIA-20> zeromq2: 03Martin Sustrik 07master * r85aa25e 10/ src/req.cpp : bug in REQ+multipart fixed -
http://bit.ly/bzWD3V[Friday 17 September 2010] [06:39:48] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:46:52] Quit omarkj has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:46:53] Nick
omarkj_ is now known as omarkj.
[Friday 17 September 2010] [06:47:41] Quit pieterh has left this server (Quit: Leaving).
[Friday 17 September 2010] [06:50:32] Quit mato has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [06:58:32] <CIA-20> zeromq2: 03Martin Sustrik 07master * rc7b9ba3 10/ (src/command.hpp src/session.cpp): reconnection process fixed when failure occurs during init phase -
http://bit.ly/cIBUcl[Friday 17 September 2010] [06:58:58] Quit
AndrewBC_ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [08:10:38] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [08:50:47] Quit sbahra has left this server (Remote host closed the connection).
[Friday 17 September 2010] [10:05:01] Join plq has joined this channel (
~plq@78.189.30.218).
[Friday 17 September 2010] [10:09:28] Quit bgranger has left this server (Quit: bgranger).
[Friday 17 September 2010] [10:27:57] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 17 September 2010] [10:32:15] Quit plq has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [12:28:09] Quit cremes has left this server (Quit: cremes).
[Friday 17 September 2010] [12:28:48] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Friday 17 September 2010] [12:34:43] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [12:36:18] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [13:29:54] Quit yrashk has left this server (Ping timeout: 246 seconds).
[Friday 17 September 2010] [13:39:04] Quit omarkj has left this server (Quit: omarkj).
[Friday 17 September 2010] [14:48:58] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [14:51:00] Quit __moore__ has left this server (Remote host closed the connection).
[Friday 17 September 2010] [14:55:33] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [15:00:22] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [15:05:32] Quit kelvink has left this server (Quit: Leaving.).
[Friday 17 September 2010] [16:02:02] <bgranger> sustrik: question about msg ref counting...
[Friday 17 September 2010] [16:11:22] <keffo> I think they’re all off work :)
[Friday 17 September 2010] [16:11:32] <keffo> starting to think I’m the only one around anymore :)
[Friday 17 September 2010] [16:12:29] <keffo> lol, last time anyone even spoke in here was about 11h ago :)
[Friday 17 September 2010] [16:12:39] <andrewvc> and you just broke the record
[Friday 17 September 2010] [16:12:55] <keffo> no, bgranger did 10 mins ago :)
[Friday 17 September 2010] [16:14:09] <andrewvc> oh damn, hehe, that’s what I get for sleeping my laptop, missing all the action
[Friday 17 September 2010] [16:16:43] <keffo> :)
[Friday 17 September 2010] [16:17:16] <keffo> Does anyone know best-practices for transferring large amounts of data? Simply to split into multipart?
[Friday 17 September 2010] [16:17:19] Quit yrashk has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [16:17:46] <andrewvc> I remember there were benchmarks on the zeromq site with various message sizes, ranging up to something huge
[Friday 17 September 2010] [16:18:22] <keffo> right, but simply sending one huge message just sounds like mem hog to me
[Friday 17 September 2010] [16:29:58] Quit xla has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [16:32:34] <sustrik> i am back
[Friday 17 September 2010] [16:32:49] <sustrik> bgranger: what’s the question?
[Friday 17 September 2010] [16:34:24] <sustrik> keffo: so is multipart message, you have to break it into sequence of messages
[Friday 17 September 2010] [16:34:42] <sustrik> then you have to define your own transfer protocol etc.
[Friday 17 September 2010] [16:34:45] <sustrik> it’s not easy
[Friday 17 September 2010] [16:34:51] <keffo> mm
[Friday 17 September 2010] [16:35:01] <sustrik> what do you want to do?
[Friday 17 September 2010] [16:35:31] <keffo> To be honest I’m leaning towards an accompanying torrent lib for larger transfers
[Friday 17 September 2010] [16:35:46] <keffo> I solved both my deadlock and garbled data btw..
[Friday 17 September 2010] [16:35:59] <keffo> as expected it was all me :)
[Friday 17 September 2010] [16:36:03] <keffo> CRT mismatch actually
[Friday 17 September 2010] [16:36:06] <sustrik> :)
[Friday 17 September 2010] [16:36:34] <sustrik> yeah, makes sense, you would have to implement torrent kind of behaiour yourself on top of 0mq
[Friday 17 September 2010] [16:36:55] <bgranger> sustrik: we have a very subtle question about ref counting of msgs...
[Friday 17 September 2010] [16:37:16] <keffo> sustrik, I was thinking more along the lines of using the transmission codebase, or libtorrent..
[Friday 17 September 2010] [16:37:57] <sustrik> bgranger: yes?
[Friday 17 September 2010] [16:38:00] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [16:38:10] <bgranger> Hi
[Friday 17 September 2010] [16:38:11] <sustrik> keffo: sure
[Friday 17 September 2010] [16:38:14] <sustrik> hi
[Friday 17 September 2010] [16:38:22] <bgranger> We are trying to understand the refcounting
[Friday 17 September 2010] [16:38:40] <bgranger> We see that zmq_msg_close is the place where the ffn is called if the ref count is 0
[Friday 17 September 2010] [16:39:10] <bgranger> What exactly does zmq_send do with the ref count. What is zmq_close is called before a msg is actually send?
[Friday 17 September 2010] [16:39:26] <bgranger> Does send somehow end up calling zmq_msg_close?
[Friday 17 September 2010] [16:42:42] <sustrik> yes, send takes ownership of your message
[Friday 17 September 2010] [16:43:03] <bgranger> OK, we just saw that at the end of xsend there is
[Friday 17 September 2010] [16:43:07] <sustrik> your message object will be empty after return from the send call
[Friday 17 September 2010] [16:43:10] <bgranger> int rc = zmq_msg_init (
msg_);
[Friday 17 September 2010] [16:43:30] <sustrik> yes, that initialises your message to empty
[Friday 17 September 2010] [16:43:31] <bgranger> So if you then call zmq_msg_close you are not closing the original msg, but a new empty one.
[Friday 17 September 2010] [16:43:38] <bgranger> Great!
[Friday 17 September 2010] [16:43:39] <sustrik> exactly
[Friday 17 September 2010] [16:43:59] <bgranger> We are getting python to know when all copies of a message are done being sent.
[Friday 17 September 2010] [16:44:29] <sustrik> intercepting the free function?
[Friday 17 September 2010] [16:44:41] <bgranger> There is a subtle issue with non-copy sends. The user has to be able to know when zmq is done with it and they can edit the buffer again.
[Friday 17 September 2010] [16:44:46] <bgranger> Yes, we use the ffn.
[Friday 17 September 2010] [16:45:13] <bgranger> Otherwise the user might accidentally start the edit the buffer they sent before zmq has send it...
[Friday 17 September 2010] [16:45:29] <bgranger> non-copy sends are super subtle...
[Friday 17 September 2010] [16:45:34] <sustrik> so you are basically writing your own allocator
[Friday 17 September 2010] [16:46:21] <bgranger> We use Pythons garbage collection mechanism along with our own ref counts that allow us to track the underlying zmq ref counts.
[Friday 17 September 2010] [16:46:45] <sustrik> what’s the goal?
[Friday 17 September 2010] [16:47:12] <bgranger> In Python you might have a mutable array of numbers that you want to send using non-copy.
[Friday 17 September 2010] [16:47:27] <bgranger> You send it, and maybe then you want to modify it.
[Friday 17 September 2010] [16:47:37] <sustrik> i see
[Friday 17 September 2010] [16:47:40] <bgranger> But if you modify it before zmq sends it, the wrong thing gets sent
[Friday 17 September 2010] [16:47:54] <bgranger> This is common in Python with numerical arrays (matrices, etc.)
[Friday 17 September 2010] [16:48:23] <bgranger> For large static data it is not a problem, but in numerical/scientific computing, most data structures are mutable...
[Friday 17 September 2010] [16:49:08] <bgranger> sustrik: I think we understand the refcounting better now though. Thanks!
[Friday 17 September 2010] [16:50:55] <sustrik> np
[Friday 17 September 2010] [16:51:04] <sustrik> a question though:
[Friday 17 September 2010] [16:51:33] <sustrik> doesn’t it turn the messaging into sync model?
[Friday 17 September 2010] [16:51:43] <sustrik> one-message-a-time?
[Friday 17 September 2010] [17:04:34] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [17:08:59] <bgranger> sustrik: back from lunch...
[Friday 17 September 2010] [17:09:15] <bgranger> no the messaging is still fully async
[Friday 17 September 2010] [17:09:21] <minrk> sustrik: I’m not sure what your question means. All we did was add an object that tracks whether a message no longer has references in zmq
[Friday 17 September 2010] [17:09:48] <bgranger> You can then query that object to learn if the message has fully been sent.
[Friday 17 September 2010] [17:10:03] <bgranger> send does not block any more than it used to
[Friday 17 September 2010] [17:10:29] <minrk> or if there are any message objects pointing to it at all
[Friday 17 September 2010] [17:11:15] <bgranger> minrk: it would be nice if we could have a method of MessageTracker that would block until the msg is not pending though.
[Friday 17 September 2010] [17:12:56] <minrk> that’s easy
[Friday 17 September 2010] [17:13:34] Quit keffo has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [17:13:42] <bgranger> minrk: I am ready when you are....just give me a call on skype
[Friday 17 September 2010] [17:13:50] <minrk> okay
[Friday 17 September 2010] [17:14:08] <minrk> give me a few minutes
[Friday 17 September 2010] [17:15:27] <bgranger> no problem...
[Friday 17 September 2010] [17:20:09] <minrk> okay
[Friday 17 September 2010] [17:22:24] <mrossi> anyone know who is on the sending side of the signaler instance inside of the app_thread class?
[Friday 17 September 2010] [17:24:07] <mrossi> have an app that occasionally gets hung at line 88 of app_thread.cpp doing a signaler.recv()
[Friday 17 September 2010] [17:38:05] <dbudworth> question from a newbie zeromq user. I’d like to implement a socket server type system where the server is stateful for each client connection. Is there a zeromq socket type for this? it appears as though the server doesn’t have any context about which client it’s sending to at any one time
[Friday 17 September 2010] [17:46:16] Quit mrossi has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [18:09:29] Quit gavinstark has left this server (Remote host closed the connection).
[Friday 17 September 2010] [18:17:04] Quit
miguelito__ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [18:18:02] Quit skaar has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [18:34:58] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [18:35:02] Part minrk has left this channel.
[Friday 17 September 2010] [18:36:53] Quit dbudworth has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [18:51:56] <ModusPwnens> Hmm, can anyone take a look at some results from benchmarking tests?
[Friday 17 September 2010] [18:52:16] <ModusPwnens> I am seeing some strange results, but maybe they are normal
[Friday 17 September 2010] [19:25:30] Quit bgranger has left this server (Read error: Operation timed out).
[Friday 17 September 2010] [19:40:28] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Friday 17 September 2010] [19:48:48] Quit rossij has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [20:05:09] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [20:46:37] Quit jhawk28 has left this server (Remote host closed the connection).
[Friday 17 September 2010] [20:50:24] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [20:52:20] Quit bennymack has left this server (Write error: Broken pipe).
[Friday 17 September 2010] [21:15:43] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Friday 17 September 2010] [21:27:38] Quit ModusPwnens has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [21:50:26] Quit yrashk has left this server (Ping timeout: 264 seconds).
[Friday 17 September 2010] [22:10:08] Quit chowmeined has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [22:14:32] Quit chowmeined has left this server (Client Quit).
[Friday 17 September 2010] [23:01:11] Quit omarkj has left this server (Quit: omarkj).
[Friday 17 September 2010] [23:08:31] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:25:41] Quit
aleator_ has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [23:27:02] Quit bgranger has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:27:02] Quit bennymac1 has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:27:02] Quit Tasser has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:38:47] <benford> good morning folks. can anyone answer a question about non blocking recieve and send?
[Friday 17 September 2010] [23:45:02] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [00:02:58] Quit beppu has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [00:03:18] Quit skaar has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [00:19:11] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Saturday 18 September 2010] [01:42:31] Quit yrashk has left this server (Ping timeout: 255 seconds).
[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0c215fa 10/ (src/fq.cpp src/lb.cpp src/socket_base.cpp): potential duplicate termination of pipes removed -
http://bit.ly/9s1gJt[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rfb6ce53 10/ src/own.cpp : more details on launch_sibling in comments -
http://bit.ly/bQt0hE[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r4c6d07d 10/ (15 files): single term ack counting mechanism for every socket (no separate mechanisms for fq_t and lb_t) -
http://bit.ly/cS50VY[Saturday 18 September 2010] [02:17:24] <sustrik> benford: shoot
[Saturday 18 September 2010] [02:17:33] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Saturday 18 September 2010] [02:18:44] <benford> hi, i think it was user error :-) i’m trying to wrap the concurrent eventlet library around socket recv/send and was having some tryouble with polling
[Saturday 18 September 2010] [02:20:03] <sustrik> ok
[Saturday 18 September 2010] [02:29:55] <benford> actually - i guess i’ve got another question: if i’m doing a zmq_poll what might throw errno 14 ‘Bad address’
[Saturday 18 September 2010] [02:30:36] <benford> i’m probably calling this with an int representing an eventlet socket so i’m thinking that the flags are causing some sort of error?
[Saturday 18 September 2010] [02:30:57] <benford> (this is in pyzmq btw)
[Saturday 18 September 2010] [02:34:21] Quit bgranger has left this server (Quit: bgranger).
[Saturday 18 September 2010] [02:34:37] <benford> when using zmq_poll do i still have to stick with the POLLIN and POLLOUT flags or would i need to replace them with the equvalent select flags?
[Saturday 18 September 2010] [02:37:34] <benford> doh - figured it - was using a different context in the same thread
[Saturday 18 September 2010] [02:45:29] Quit mikko has left this server (Remote host closed the connection).
[Saturday 18 September 2010] [03:02:32] Quit kleppari has left this server (Read error: Operation timed out).
[Saturday 18 September 2010] [03:11:13] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Saturday 18 September 2010] [05:05:21] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [05:39:22] Quit kelvink has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [05:51:08] Quit yrashk has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [05:55:50] Join yrashk has joined this channel (
~yrashk@70.28.245.1).
[Saturday 18 September 2010] [06:05:08] Quit benford has left this server (Quit: benford).
[Saturday 18 September 2010] [06:08:04] Quit xla has left this server (Ping timeout: 272 seconds).
[Saturday 18 September 2010] [06:24:34] Quit mitkok has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [06:41:30] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [06:42:54] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Saturday 18 September 2010] [06:43:58] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [07:40:04] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [08:25:40] Quit xla has left this server (Quit: leaving).
[Saturday 18 September 2010] [09:06:58] Quit plq has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [09:20:20] Quit keffo has left this server (Quit: Leaving).
[Saturday 18 September 2010] [09:21:27] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [10:11:13] Quit benford has left this server (Client Quit).
[Saturday 18 September 2010] [10:14:33] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 18 September 2010] [10:17:41] Quit psino has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [10:45:41] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [11:25:25] Part mr19 has left this channel.
[Saturday 18 September 2010] [11:29:48] Quit mitkok has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [12:08:35] Quit plq has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [12:12:08] Quit bgranger has left this server (Quit: bgranger).
[Saturday 18 September 2010] [12:20:41] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [12:37:13] Quit rossij has left this server (Ping timeout: 245 seconds).
[Saturday 18 September 2010] [13:08:53] Quit xla has left this server (Ping timeout: 245 seconds).
[Saturday 18 September 2010] [13:15:58] Quit benford has left this server (Quit: benford).
[Saturday 18 September 2010] [13:52:50] Quit benford has left this server (Client Quit).
[Saturday 18 September 2010] [14:16:48] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [15:24:28] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [15:28:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [15:30:08] Quit ModusPwnens has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [15:32:21] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [16:01:51] <mr19> docs say to call zmq_init() from the “start of your main line of code”, my app starts a new thread that calls zmq_init() (via the zmq::context_t constructor) at thread startup. can easily be switched but curious as to whether this can cause problems. quick look through the code didn’t raise any flags to me.
[Saturday 18 September 2010] [16:06:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Saturday 18 September 2010] [16:34:20] <sustrik> mr19: to be able to communicate in inter-thread manner you have t share the context
[Saturday 18 September 2010] [16:34:42] <sustrik> i mean inproc:// transport
[Saturday 18 September 2010] [17:15:25] Quit sbahra has left this server (Ping timeout: 252 seconds).
[Saturday 18 September 2010] [17:27:14] <mr19> ok, i’m just using tcp and only from within that thread, just trying to simplify my code – i’m going to move the ctx instantiation to the main thread but was just curious if there are any other reasons for requiring the call to zmq_init() in the main thread
[Saturday 18 September 2010] [17:33:01] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [18:23:34] Quit sbahra has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [18:28:01] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [18:33:34] Quit psino has left this server (Client Quit).
[Saturday 18 September 2010] [19:44:12] Quit kelvink has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [19:47:08] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [20:11:50] Quit bbigras has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [21:00:12] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Saturday 18 September 2010] [22:16:16] Part mr19 has left this channel.
[Saturday 18 September 2010] [23:35:34] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Saturday 18 September 2010] [23:57:08] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [00:41:43] <dbudworth> is there a socket type that’s usable for request/streaming response type communications? ie: N clients open connection to a server and the server has a unique conversation with each client. (similar to regular socket accept/listen model)
[Sunday 19 September 2010] [01:09:28] <sustrik> if you want to control each socket, you need a regular socket
[Sunday 19 September 2010] [02:28:26] Quit bgranger has left this server (Quit: bgranger).
[Sunday 19 September 2010] [03:04:46] Quit kleppari has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [03:06:08] <CIA-20> zeromq2: 03Martin Sustrik 07master * r5153b63 10/ (src/session.cpp src/session.hpp): obsolete ‘active’ flag removed from session_t -
http://bit.ly/cRgwVE[Sunday 19 September 2010] [03:06:08] <CIA-20> zeromq2: 03Martin Sustrik 07master * r1f10208 10/ src/session.cpp : termination of pipe via delimiter message could stuck when no data were read from the pipe (because connection wasn’t active) – fixed -
http://bit.ly/dpganD[Sunday 19 September 2010] [03:06:18] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Sunday 19 September 2010] [03:12:19] Quit ikolomiets has left this server (Ping timeout: 252 seconds).
[Sunday 19 September 2010] [04:18:53] Quit mitkok has left this server (Ping timeout: 245 seconds).
[Sunday 19 September 2010] [04:51:01] Quit AndrewBC has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [05:53:38] Quit psino has left this server (Changing host).
[Sunday 19 September 2010] [05:53:41] Quit guido_g has left this server (Excess Flood).
[Sunday 19 September 2010] [06:43:04] Quit psino has left this server (Quit: psino).
[Sunday 19 September 2010] [08:00:05] Quit keffo has left this server (Ping timeout: 255 seconds).
[Sunday 19 September 2010] [09:02:38] Quit plq has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [09:22:03] Quit xla has left this server (Quit: leaving).
[Sunday 19 September 2010] [09:28:25] Quit iFire has left this server (Remote host closed the connection).
[Sunday 19 September 2010] [10:49:40] Quit psino has left this server (Quit: psino).
[Sunday 19 September 2010] [11:14:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 19 September 2010] [12:47:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 19 September 2010] [12:58:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 19 September 2010] [13:12:36] Quit keffo has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [14:02:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [14:27:33] Part sigmonsays has left this channel (“Leaving”).
[Sunday 19 September 2010] [15:15:03] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [15:21:20] Quit keffo has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [15:46:03] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [15:59:58] <CIA-20> zeromq2: 03Martin Sustrik 07master * r471013a 10/ src/zmq_init.cpp : two races fixed in session creation -
http://bit.ly/9UgZMy[Sunday 19 September 2010] [15:59:59] <CIA-20> zeromq2: 03Martin Sustrik 07master * r938689a 10/ (src/ctx.cpp src/zmq_init.cpp): synchronisation problem fixed in ctx_t -
http://bit.ly/bcX8OW[Sunday 19 September 2010] [16:04:03] Part AndrewBC has left this channel (“Leaving”).
[Sunday 19 September 2010] [16:21:47] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 19 September 2010] [16:25:35] Quit mitkok has left this server (Quit: Leaving.).
[Sunday 19 September 2010] [16:27:56] Quit rgl has left this server (Quit: Leaving).
[Sunday 19 September 2010] [16:50:14] <CIA-20> zeromq2: 03Martin Sustrik 07master * rcda3c96 10/ src/ctx.cpp : minor code reorganisation in ctx_t::terminate -
http://bit.ly/96qYvk[Sunday 19 September 2010] [16:50:16] <CIA-20> zeromq2: 03Martin Sustrik 07master * r1d23997 10/ (src/zmq_engine.cpp src/zmq_engine.hpp src/zmq_init.cpp): zmq_init_t destroyed zmq_engine_t before plugging it out from the poller first – fixed -
http://bit.ly/cvBySe[Sunday 19 September 2010] [16:50:38] Quit psino has left this server (Ping timeout: 255 seconds).
[Sunday 19 September 2010] [16:50:38] Nick
psino_ is now known as psino.
[Sunday 19 September 2010] [17:26:49] Quit plq has left this server (Remote host closed the connection).
[Sunday 19 September 2010] [17:58:28] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Sunday 19 September 2010] [18:06:05] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [18:06:23] Quit mitkok has left this server (Client Quit).
[Sunday 19 September 2010] [18:07:37] <CIA-20> zeromq2: 03Martin Sustrik 07master * r328c92a 10/ (4 files): problem with engine being attached to session while it’s being terminated fixed -
http://bit.ly/aVsix0[Sunday 19 September 2010] [19:15:32] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [20:01:47] Quit carrus85 has left this server (Quit: Leaving).
[Sunday 19 September 2010] [20:07:30] Quit
miguelito_ has left this server (Ping timeout: 252 seconds).
[Sunday 19 September 2010] [20:39:59] Quit miguelito has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [23:06:30] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Sunday 19 September 2010] [23:20:05] <stoneman> hello
[Sunday 19 September 2010] [23:20:10] Join dirtmcgirt has joined this channel (
~dirtmcgir@2002:62f8:7ff4:0:daa2:5eff:fe98:8f1c).
[Sunday 19 September 2010] [23:20:30] <stoneman> is anyone here?
[Sunday 19 September 2010] [23:21:36] Quit dirtmcgirt has left this server (Client Quit).
[Sunday 19 September 2010] [23:27:17] <dbudworth> Trying to figure out if 0mq is appropriate for my project, simple description would be something like a sticky load balancer. once a client talks to a service, it sticks with that one. for a bi-directional conversation between 2 nodes once an initial node has been selected. ie: my client does round robin on each new conversation but updates to a given conversation have to stick to the beginning server
[Sunday 19 September 2010] [23:27:43] <dbudworth> and hello stoneman, i’m here. but basically not useful if you are looking for help
[Sunday 19 September 2010] [23:35:48] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [23:36:07] Part stoneman has left this channel.
[Monday 20 September 2010] [01:08:51] Quit guido_g has left this server (Quit: Boom...).
[Monday 20 September 2010] [01:33:38] Quit jsimmons has left this server (Ping timeout: 264 seconds).
[Monday 20 September 2010] [01:56:23] Quit keffo has left this server (Ping timeout: 245 seconds).
[Monday 20 September 2010] [02:22:21] Quit benford has left this server (Quit: benford).
[Monday 20 September 2010] [02:54:04] Part benford has left this channel.
[Monday 20 September 2010] [03:04:31] Quit kleppari has left this server (Ping timeout: 246 seconds).
[Monday 20 September 2010] [03:06:28] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Monday 20 September 2010] [04:22:49] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 20 September 2010] [05:10:23] <mato> sustrik: i was just thinking about the zmq_term()/zmq_close() semantics
[Monday 20 September 2010] [05:10:40] <mato> sustrik: and i came up with one small detail
[Monday 20 September 2010] [05:11:03] <mato> sustrik: which may or may not be helpful in the case where zmq_term() would wait indefinitely on exit
[Monday 20 September 2010] [05:11:23] <mato> sustrik: the thing is, syscall close() means “commit”.
[Monday 20 September 2010] [05:12:02] <mato> sustrik: if you don’t close your filedescriptors, and don’t use fsync() (not for sockets obviously), then there is no commitment on the part of the kernel
[Monday 20 September 2010] [05:13:02] <mato> sustrik: hence, as far as sockets with data in-flight at zmq_term() time are concerned, we only need care about those that zmq_close() has actually been called on
[Monday 20 September 2010] [05:13:10] <mato> sustrik: for any other sockets, all bets are off
[Monday 20 September 2010] [05:19:00] <sustrik> mato: sure, but user has to close all the sockets anyway
[Monday 20 September 2010] [05:19:12] <sustrik> otherwise he’ll end up with memory leaks
[Monday 20 September 2010] [05:19:52] <mato> sustrik: i was thinking more of the case of “the application terminating in an abnormal way”
[Monday 20 September 2010] [05:20:03] <sustrik> no guarantees then
[Monday 20 September 2010] [05:20:14] <mato> sure, what i mean is...
[Monday 20 September 2010] [05:20:24] <mato> if an application handles e.g. SIGINT/SIGTERM
[Monday 20 September 2010] [05:20:39] <mato> should it go and call zmq_term() as part of its exit process in such a case?
[Monday 20 September 2010] [05:20:48] <mato> or explicitly not do that, becuase the call may block...?
[Monday 20 September 2010] [05:21:02] <mato> i guess the answer is application dependent
[Monday 20 September 2010] [05:21:21] <sustrik> no idea
[Monday 20 September 2010] [05:21:48] <mato> oh, and another thing
[Monday 20 September 2010] [05:21:58] <mato> we talked about the non-zero-copy APIs
[Monday 20 September 2010] [05:22:10] <mato> and i have the most obvious names :)
[Monday 20 September 2010] [05:22:22] <mato> zmq_sendcopy and zmq_recvcopy ... ?
[Monday 20 September 2010] [05:22:43] <mato> would do fine for 2.1, in 3.x the whole mess can be done the right way
[Monday 20 September 2010] [05:23:31] <sustrik> dunno, does it makes sense to introduce something that will be changed anyway?
[Monday 20 September 2010] [05:23:45] <sustrik> it’s just asking for problems
[Monday 20 September 2010] [05:23:50] <mato> why?
[Monday 20 September 2010] [05:24:41] <mato> well, one reason is that pieter uses all these “helper” functions in the user guide precisely for this reason
[Monday 20 September 2010] [05:25:07] <sustrik> so, we’ll have send, sendcopy and the helper function
[Monday 20 September 2010] [05:25:16] <sustrik> :)
[Monday 20 September 2010] [05:25:16] <mato> no, the helpers can go away
[Monday 20 September 2010] [05:25:36] <mato> check with pieter, but imo the reason those are there is becuase zerocopy in C is “too much typing” or something :)
[Monday 20 September 2010] [05:25:43] <mato> at least that’s what he told me
[Monday 20 September 2010] [05:25:54] <sustrik> anyway, first thing on this path is to defined the semantics for recvcopy when the buffer is no large enough
[Monday 20 September 2010] [05:26:06] * sustrik has to have a look at sctp
[Monday 20 September 2010] [05:27:10] <mato> yeah, good point
[Monday 20 September 2010] [05:27:28] <mato> presumably some error will have to be returned ...
[Monday 20 September 2010] [05:27:47] <mato> SCTP works in terms of messages as atomic units?
[Monday 20 September 2010] [05:29:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [05:30:28] <sustrik> mato: yes
[Monday 20 September 2010] [08:05:11] Quit mitkok has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [08:14:24] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [08:16:38] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 20 September 2010] [09:03:10] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [09:03:58] Quit dbudworth has left this server (Quit: dbudworth).
[Monday 20 September 2010] [09:25:09] Quit guido_g has left this server (Quit: Boom...).
[Monday 20 September 2010] [09:36:51] <user> was hoping someone can help me
[Monday 20 September 2010] [09:37:43] <user> cant download anything always comes up with an error
[Monday 20 September 2010] [09:44:18] <user> whats that?
[Monday 20 September 2010] [09:44:43] <drbobbeaty> The POSIX version is the version for Linux, etc. There’s also a Windows version on the same page.
[Monday 20 September 2010] [09:44:47] <mato> sustrik: are you around?
[Monday 20 September 2010] [09:45:20] <mato> sustrik: I’ve found a silly bug in the select() impl. of zmq_poll() on master
[Monday 20 September 2010] [09:45:31] <sustrik> mato: here am I
[Monday 20 September 2010] [09:45:35] <mato> sustrik: trivial fix, but something else is broken when using select()
[Monday 20 September 2010] [09:45:44] <user> k. my error code tells me: Archive: /media/OFFICE12/setup.exe
[Monday 20 September 2010] [09:45:44] <user> [/media/OFFICE12/setup.exe]
[Monday 20 September 2010] [09:45:44] <user> End-of-central-directory signature not found. Either this file is not
[Monday 20 September 2010] [09:45:44] <user> a zipfile, or it constitutes one disk of a multi-part archive. In the
[Monday 20 September 2010] [09:45:44] <user> latter case the central directory and zipfile comment will be found on
[Monday 20 September 2010] [09:45:45] <user> the last disk(s) of this archive.
[Monday 20 September 2010] [09:45:47] <user> note: /media/OFFICE12/setup.exe may be a plain executable, not an archive
[Monday 20 September 2010] [09:45:49] <user> zipinfo: cannot find zipfile directory in one of /media/OFFICE12/setup.exe or
[Monday 20 September 2010] [09:45:51] <user> /media/OFFICE12/setup.exe.zip, and cannot find /media/OFFICE12/setup.exe.ZIP, period.
[Monday 20 September 2010] [09:46:25] <mato> user: I’m sorry, you’re probably on the wrong chat room here.
[Monday 20 September 2010] [09:46:38] <user> where do i need to go?
[Monday 20 September 2010] [09:47:15] <sustrik> :)
[Monday 20 September 2010] [09:47:46] <sustrik> mato: wull?
[Monday 20 September 2010] [09:48:02] <mato> sustrik: wull?
[Monday 20 September 2010] [09:48:11] <sustrik> well?
[Monday 20 September 2010] [09:48:17] <mato> hang on
[Monday 20 September 2010] [09:48:20] <mato> i’m on bloody win32
[Monday 20 September 2010] [09:48:25] <mato> everything is confusing :-)
[Monday 20 September 2010] [09:48:28] <mato> patience...
[Monday 20 September 2010] [09:49:28] <mato> sustrik: ok, 1st, somewhee around line 547 of zmq.cpp, the select () call needs to be changed to use maxfd + 1
[Monday 20 September 2010] [09:49:32] <mato> sustrik: not just maxfd
[Monday 20 September 2010] [09:49:37] <mato> sustrik: that’s probably my fault
[Monday 20 September 2010] [09:49:43] <sustrik> ok
[Monday 20 September 2010] [09:50:26] <mato> sustrik: now, then, what i’m seeing is on win32 _or_ on Linux with ZMQ_FORCE_SELECT (and I patched zmq.cpp to also use select() for zmq_poll() when zmq_force_select() is defined)
[Monday 20 September 2010] [09:50:52] <mato> sustrik: for some reason a socket is not becoming ready on the app side when it should
[Monday 20 September 2010] [09:51:34] <mato> sustrik: i.e. data gets sent down XREP, XREQ on the client side never becomes ready
[Monday 20 September 2010] [09:51:58] <sustrik> do you have a simple test program?
[Monday 20 September 2010] [09:52:06] <mato> working on it...
[Monday 20 September 2010] [09:52:52] <sustrik> ok
[Monday 20 September 2010] [09:54:35] Part user has left this channel (“Leaving”).
[Monday 20 September 2010] [10:20:07] <mato> sustrik: ok, i have a test case... will msg you, it;s on the test box
[Monday 20 September 2010] [10:45:13] <CIA-20> zeromq2: 03Martin Lucina 07master * r1abfc92 10/ src/zmq.cpp : minor problem in zmq_poll (select version) fixed -
http://bit.ly/c9Sdnb[Monday 20 September 2010] [10:55:34] <CIA-20> zeromq2: 03Martin Lucina 07master * rf49b77e 10/ src/zmq.cpp : zmq_poll honours ZMQ_FORCE_POLL and ZMQ_FORCE_SELECT options -
http://bit.ly/dzi76e[Monday 20 September 2010] [10:56:23] Quit GeekGod has left this server (Quit: GeekGod).
[Monday 20 September 2010] [11:16:12] <sustrik> mato: afaics the problem is that ZMQ_FD is edge-trigerred
[Monday 20 September 2010] [11:17:26] <sustrik> thus, IN/OUT flag may be set in the past, but the select/poll won’t exit because of it
[Monday 20 September 2010] [11:18:32] <sustrik> mato: check how poll version of zmq_poll works
[Monday 20 September 2010] [11:20:32] <mato> sustrik: the code seems equivalent, no?
[Monday 20 September 2010] [11:20:36] <sustrik> nope
[Monday 20 September 2010] [11:21:02] <mato> then i don’t understand the problem...
[Monday 20 September 2010] [11:21:05] <sustrik> you have to delete lines 572.577
[Monday 20 September 2010] [11:21:11] <sustrik> let me do it
[Monday 20 September 2010] [11:21:58] <mato> which lines, i have different line numbers here...
[Monday 20 September 2010] [11:22:12] <mato> and i’d like to understand what the problem is
[Monday 20 September 2010] [11:22:46] <sustrik> you should _not_ check POLLIN is set on ZMQ_FD and check ZMQ_EVENTS anyway
[Monday 20 September 2010] [11:22:56] <sustrik> whather
[Monday 20 September 2010] [11:22:59] <sustrik> ehether
[Monday 20 September 2010] [11:23:01] <sustrik> whether
[Monday 20 September 2010] [11:23:02] <sustrik> :)
[Monday 20 September 2010] [11:23:03] <mato> ?
[Monday 20 September 2010] [11:23:16] <sustrik> ...whether POLLIN is set...
[Monday 20 September 2010] [11:23:48] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [11:24:44] <mato> sustrik: hmm, so you’re saying that each ZMQ_FD needs to be checked every time you come out of the select/poll() ?
[Monday 20 September 2010] [11:24:48] <CIA-20> zeromq2: 03Martin Sustrik 07master * r4d51a52 10/ src/zmq.cpp : zmq_poll (select version) now correctly assumes that ZMQ_FD is edge-trigerred -
http://bit.ly/dklauP[Monday 20 September 2010] [11:25:07] <sustrik> yes
[Monday 20 September 2010] [11:25:15] <sustrik> committed
[Monday 20 September 2010] [11:25:23] <sustrik> the test program seems to work
[Monday 20 September 2010] [11:26:23] <mato> i still don’t understand... select would not have exited if the fd did not become ready...?
[Monday 20 September 2010] [11:26:41] Quit jsimmons has left this server (Ping timeout: 276 seconds).
[Monday 20 September 2010] [11:27:26] <sustrik> what?
[Monday 20 September 2010] [11:27:51] <mato> sustrik: if you’re sitting in select(), and the notify fd becomes ready, then you read the events
[Monday 20 September 2010] [11:27:59] <mato> sustrik: what is the other code path?
[Monday 20 September 2010] [11:28:18] <sustrik> the commands in ZMQ_FD were already processed before calling zmq_poll
[Monday 20 September 2010] [11:28:25] <sustrik> select blocks forever
[Monday 20 September 2010] [11:28:38] <sustrik> although there are messages in available
[Monday 20 September 2010] [11:28:40] <mato> processed by who?
[Monday 20 September 2010] [11:28:51] <sustrik> random previous command
[Monday 20 September 2010] [11:29:12] <mato> ah, right, this is because ZMQ_FD is tapping straight into the signaller
[Monday 20 September 2010] [11:29:15] <mato> mumble
[Monday 20 September 2010] [11:29:20] <sustrik> ack
[Monday 20 September 2010] [11:29:22] <mato> i wish we could fix that
[Monday 20 September 2010] [11:29:27] <sustrik> ?
[Monday 20 September 2010] [11:29:33] <sustrik> it’s fixed
[Monday 20 September 2010] [11:29:42] <mato> this is also why you do that first_pass thing, right?
[Monday 20 September 2010] [11:29:47] <sustrik> yes
[Monday 20 September 2010] [11:29:55] <sustrik> no timout on first pass
[Monday 20 September 2010] [11:30:04] <sustrik> exit immediately
[Monday 20 September 2010] [11:30:15] <sustrik> then check whether events are availalbe
[Monday 20 September 2010] [11:30:25] <mato> this is to pick up events coming from previously processed commands, right?
[Monday 20 September 2010] [11:30:34] <sustrik> yes
[Monday 20 September 2010] [11:30:40] <mato> ok, understood
[Monday 20 September 2010] [11:31:00] * mato will re-test and see what happens
[Monday 20 September 2010] [11:38:39] <sustrik> mato: btw, if you want to optimise it you can still perform the check when !first_pass
[Monday 20 September 2010] [11:39:16] <sustrik> getting ZMQ_EVENTS can be rather slow as it involves reading from the signaler => recv()
[Monday 20 September 2010] [11:40:40] <mato> right, that might be a good idea
[Monday 20 September 2010] [11:40:48] <mato> also, is getting ZMQ_FD slow?
[Monday 20 September 2010] [11:41:01] <mato> since that is also done more times than strictly necessary
[Monday 20 September 2010] [11:41:13] <mato> sustrik: btw, the signaler uses socketpair?
[Monday 20 September 2010] [11:41:32] <mato> sustrik: which translates to a tcp connection with winsock?
[Monday 20 September 2010] [11:44:30] <sustrik> yes
[Monday 20 September 2010] [11:44:54] <mato> interesting, i get an occasional “Address already in use” on Win32 from signaler.cpp:80
[Monday 20 September 2010] [11:45:06] <sustrik> let me see
[Monday 20 September 2010] [11:45:18] <mato> I guess this is just the poor M$ WIn$ock running out of tcp ports or something
[Monday 20 September 2010] [11:45:27] <mato> it’s the “connect to remote peer” call...
[Monday 20 September 2010] [11:45:34] <sustrik> EADDRINUSE on connect?
[Monday 20 September 2010] [11:45:52] <mato> yeah
[Monday 20 September 2010] [11:45:56] <mato> bizarre
[Monday 20 September 2010] [11:46:04] <sustrik> hm, it’s documented
[Monday 20 September 2010] [11:46:16] <mato> “not enough ports” ?
[Monday 20 September 2010] [11:47:01] <sustrik> very useful description:
[Monday 20 September 2010] [11:47:02] <sustrik> [EADDRINUSE]
[Monday 20 September 2010] [11:47:02] <sustrik> Attempt to establish a connection that uses addresses that are already in use.
[Monday 20 September 2010] [11:47:30] Quit xla has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [11:47:33] <sustrik> (that’s POSIX)
[Monday 20 September 2010] [11:48:00] <mato> sustrik: windoze is slightly different
[Monday 20 September 2010] [11:48:05] <sustrik> Linux: EADDRINUSE
[Monday 20 September 2010] [11:48:05] <sustrik> Local address is already in use.
[Monday 20 September 2010] [11:49:17] <sustrik> The socket’s local address is already in use and the socket was not marked to allow address reuse with SO_REUSEADDR. This error usually occurs when executing bind, but could be delayed until the connect function if the bind was to a wildcard address (INADDR_ANY or in6addr_any) for the local IP address. A specific address needs to be implicitly bound by the connect function.
[Monday 20 September 2010] [11:49:40] <mato> Yeah, I never quite understood the SO_REUSEADDR semantics on win32
[Monday 20 September 2010] [11:49:47] <sustrik> does it make any sense to you?
[Monday 20 September 2010] [11:49:54] <mato> but i’ll try adding that in and see if it changes anything
[Monday 20 September 2010] [11:50:19] <mato> well, i think what it’s saying is “the wildcard bind() picked a port that is still in time-wait state”
[Monday 20 September 2010] [11:50:27] <mato> or some nonsense like that
[Monday 20 September 2010] [11:50:31] <sustrik> :|
[Monday 20 September 2010] [11:50:42] <mato> let me try adding SO_REUSEADDR for win32 to the signaler listen socket and see what happens
[Monday 20 September 2010] [11:50:57] <mato> i don’t care what it actually does on win32 as long as the error goes away :-)
[Monday 20 September 2010] [11:51:25] <mato> proper win32 solution is obviously to use named pipes/win32 objects/whatsits and iocp
[Monday 20 September 2010] [11:54:09] <sustrik> ok
[Monday 20 September 2010] [12:07:35] <mato> hmm
[Monday 20 September 2010] [12:07:38] <mato> i dunno
[Monday 20 September 2010] [12:07:42] <mato> doesn’t seem to help much
[Monday 20 September 2010] [12:07:47] <mato> anyway, this doesn’t really matter
[Monday 20 September 2010] [12:08:11] <mato> i added SO_REUSEADDR to both ends of the emulated socketpair and still get EADDRINUSE back
[Monday 20 September 2010] [12:08:27] <mato> so i think it’s just poor windows running out of ports to auto-assign or something :-)
[Monday 20 September 2010] [12:08:50] <mato> sustrik: anyway, the problem with select() has been fixed, so it’s all good!
[Monday 20 September 2010] [12:11:05] <sustrik> nice
[Monday 20 September 2010] [12:11:26] <sustrik> you may fill in the bug report for the EADDRINUSE stuff
[Monday 20 September 2010] [12:24:30] <mato> done
[Monday 20 September 2010] [12:45:23] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Monday 20 September 2010] [13:07:13] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Monday 20 September 2010] [13:14:11] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Monday 20 September 2010] [13:18:32] Quit psino has left this server (Quit: psino).
[Monday 20 September 2010] [13:38:12] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Monday 20 September 2010] [13:50:52] Quit mivert has left this server (Quit: Leaving).
[Monday 20 September 2010] [14:28:40] Quit omarkj has left this server (Quit: omarkj).
[Monday 20 September 2010] [14:52:12] Quit mitkok has left this server (Read error: No route to host).
[Monday 20 September 2010] [14:55:48] Quit mitkok1 has left this server (Client Quit).
[Monday 20 September 2010] [15:13:59] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 20 September 2010] [15:54:16] Quit kelvink has left this server (Quit: Leaving.).
[Monday 20 September 2010] [16:00:52] <cpscotti> Hello there, anyone up to a philosophical (although basic) discussion on a zmq networking topology for a “generic” application? Regarding many clients & services but without a broker.
[Monday 20 September 2010] [16:03:43] Quit jhawk28 has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [16:07:18] Quit mitkok has left this server (Client Quit).
[Monday 20 September 2010] [16:08:37] Nick mitkok is now known as mytrile.
[Monday 20 September 2010] [16:17:01] <cremes> cpscotti: i’d be happy to have that conversation with you tomorrow; i have to leave for the rest of today
[Monday 20 September 2010] [16:19:20] <cpscotti> cremes: thanks.. if nothing is solved in my side of things I’ll try tomorrow then
[Monday 20 September 2010] [16:19:50] <cremes> ok
[Monday 20 September 2010] [16:32:22] Quit mytrile has left this server (Read error: No route to host).
[Monday 20 September 2010] [16:34:27] Quit mitkok has left this server (Client Quit).
[Monday 20 September 2010] [17:07:39] <ModusPwnens> is the send function faster than the receive function? or are they both just as fast?
[Monday 20 September 2010] [17:08:10] <cpscotti> send doesn’t block, recv blocks
[Monday 20 September 2010] [17:08:21] <cpscotti> (as for the “program flow” speed)
[Monday 20 September 2010] [17:08:33] <cpscotti> now for the underlying stuff, dunno
[Monday 20 September 2010] [17:08:36] <ModusPwnens> even with pub/sub topology?
[Monday 20 September 2010] [17:08:37] <jhawk28> recv can do a noblock flag
[Monday 20 September 2010] [17:08:49] <cpscotti> awl.. yep..
[Monday 20 September 2010] [17:08:56] <ModusPwnens> I see.
[Monday 20 September 2010] [17:09:09] <ModusPwnens> So that would explain benchmarking tests where
[Monday 20 September 2010] [17:09:12] <ModusPwnens> i run several in a row
[Monday 20 September 2010] [17:09:59] <ModusPwnens> thats probably better than explaining it
[Monday 20 September 2010] [17:10:37] <ModusPwnens> I’ve sort of run into a wall with this and have been stuck on it for several days
[Monday 20 September 2010] [17:10:44] <ModusPwnens> because I am seeing those results
[Monday 20 September 2010] [17:10:47] <ModusPwnens> and am not sure what to make of them
[Monday 20 September 2010] [18:10:12] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [18:26:19] Quit cpscotti has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [18:30:32] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 20 September 2010] [18:35:19] Quit mytrile has left this server (Quit: Leaving.).
[Monday 20 September 2010] [18:41:15] <larrytheliquid> are there any semantic implications to connecting multiple times to the same address?
[Monday 20 September 2010] [18:41:36] <larrytheliquid> from the same socket, that is
[Monday 20 September 2010] [18:44:18] Quit sbahra has left this server (Remote host closed the connection).
[Monday 20 September 2010] [18:45:20] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 20 September 2010] [18:49:06] Quit gavinstark has left this server (Remote host closed the connection).
[Monday 20 September 2010] [18:53:43] Quit dbudworth has left this server (Quit: dbudworth).
[Monday 20 September 2010] [19:02:09] Quit ModusPwnens has left this server.
[Monday 20 September 2010] [19:31:31] Quit Samy has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [19:36:02] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Monday 20 September 2010] [19:38:33] Join Samy has joined this channel (
~sbahra@8.8.38.2).
[Monday 20 September 2010] [20:58:34] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Monday 20 September 2010] [21:17:48] Quit andrewvc has left this server (Quit: andrewvc).
*** on Mon Sep 20 21:52:23 2010
[Monday 20 September 2010] [21:52:23] Topic The channel topic is “Welcome!”.
[Monday 20 September 2010] [21:52:27] Mode Channel modes: no messages from outside, topic protection
[Monday 20 September 2010] [21:52:27] Created This channel was created on 2010-02-05 10:44.
[Monday 20 September 2010] [22:15:40] Quit jhawk28 has left this server (Remote host closed the connection).
[Monday 20 September 2010] [23:10:53] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 21 September 2010] [00:22:53] Quit iFire has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [01:29:13] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [01:37:27] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 21 September 2010] [01:43:53] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [01:45:30] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Tuesday 21 September 2010] [01:55:18] Quit grahamalot has left this server (Quit: grahamalot).
[Tuesday 21 September 2010] [03:03:31] Quit benford has left this server (Quit: benford).
[Tuesday 21 September 2010] [03:04:26] Quit kleppari has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [03:10:35] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Tuesday 21 September 2010] [03:36:36] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [04:05:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * r9c11886 10/ (src/socket_base.cpp src/socket_base.hpp): The flag in the socket has clear semantics now – it tracks whether corresponding context was closed, it doesn’t track whether zmq_close was called on the socket itself -
http://bit.ly/d6ktZI[Tuesday 21 September 2010] [04:05:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * re478468 10/ src/socket_base.hpp : minor error in comment fixed -
http://bit.ly/a4RuQJ[Tuesday 21 September 2010] [04:45:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [04:50:56] Quit xla has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [04:52:20] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [05:20:42] Quit xla has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [05:22:11] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [05:25:11] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [06:37:38] Quit SecretAgent has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [07:19:50] Quit timothyfitz has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [08:52:37] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [09:38:31] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [10:30:52] Quit mytrile has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [10:33:26] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 21 September 2010] [11:23:17] Part sd88g93 has left this channel.
[Tuesday 21 September 2010] [11:56:30] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [11:56:52] <starkdg> hello
[Tuesday 21 September 2010] [11:57:09] <guido_g> hi
[Tuesday 21 September 2010] [12:12:06] <CIA-20> rbzmq: 03Brian Buchanan 07master * rdc552c2 10/ rbzmq.c : Make PUSH and PULL constants work even if using an older version of the library. -
http://bit.ly/9ZisAu[Tuesday 21 September 2010] [12:12:08] <CIA-20> rbzmq: 03Brian Buchanan 07master * rf17f6ca 10/ zmq.gemspec : update gem version to 2.0.9 -
http://bit.ly/b4wU0A[Tuesday 21 September 2010] [12:21:58] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [13:02:20] Quit RobertLJ has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [13:24:48] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [13:53:42] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 21 September 2010] [13:56:46] Quit starkdg has left this server (Quit: goodbye).
[Tuesday 21 September 2010] [14:05:19] Quit xla has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [14:19:27] <AltraMayor> Hi there,
[Tuesday 21 September 2010] [14:20:06] <AltraMayor> I’ve been reading about PNUTS, a NoSQL solution written by Yahoo!
[Tuesday 21 September 2010] [14:21:21] <AltraMayor> A very important element of their design is their message brokers, which guarantee that messages aren’t lost. For example, they can make brokers replicate messages with others to support failure.
[Tuesday 21 September 2010] [14:21:45] <AltraMayor> Is this use case supported by zeromq?
[Tuesday 21 September 2010] [14:21:55] <guido_g> no, zeromq doesn’t have that
[Tuesday 21 September 2010] [14:22:34] <AltraMayor> Thanks, guido_g.
[Tuesday 21 September 2010] [14:23:21] <AltraMayor> I’ve read that page, but since it says that’s more coming, I couldn’t assume that it’s all.
[Tuesday 21 September 2010] [14:23:54] <AltraMayor> Does anyone know a message broker designed to support that level of failure?
[Tuesday 21 September 2010] [14:24:09] <guido_g> you m ight have skipped the important things
[Tuesday 21 September 2010] [14:24:31] <AltraMayor> I don’t follow you, guido_g.
[Tuesday 21 September 2010] [14:24:33] <guido_g> like that mq doesn’t have or is a message broker
[Tuesday 21 September 2010] [14:25:11] <AltraMayor> I understand that zeromq isn’t a broker, but its durable option isn’t enough.
[Tuesday 21 September 2010] [14:25:41] <cremes> AltraMayor: the roadmap is a wiki page that can be edited by anyone who signs up for an account
[Tuesday 21 September 2010] [14:25:52] <cremes> if you think there are things missing, feel free to add them
[Tuesday 21 September 2010] [14:25:55] <AltraMayor> I also understood I could implement a broker as reliable as I want using zeromq, but this would be just a piece of what I’m looking for.
[Tuesday 21 September 2010] [14:29:12] <AltraMayor> Cremes: what’s the link to the roadmap page you mentioned?
[Tuesday 21 September 2010] [14:31:55] <AltraMayor> Cremes: do you think there’s interest among the developers to implement this level of robustness? As I read, zeroqm is implemented by iMatix to meet their demands, I don’t know if they’d be interested in a wish list of a guy that wants to have an open-source alternative to PNUTS.
[Tuesday 21 September 2010] [14:32:25] <cremes> good point... i heard their ceo speak on that very topic yesterday
[Tuesday 21 September 2010] [14:32:34] <cremes> if you write the code, it will get done
[Tuesday 21 September 2010] [14:32:41] <cremes> if you pay them to write the code, it will get done
[Tuesday 21 September 2010] [14:35:03] <AltraMayor> I understand, but I’m looking for pieces to estimate the total effort to have an opensource PNUTS.
[Tuesday 21 September 2010] [14:36:37] <cremes> okay, well i don’t know what pnuts is :)
[Tuesday 21 September 2010] [14:39:57] <AltraMayor> Cremes: can you provide me a small description how you use 0mq?
[Tuesday 21 September 2010] [14:41:04] <cremes> AltraMayor: sure...
[Tuesday 21 September 2010] [14:41:24] <cremes> i use it to tie together a bunch of distributed components; i used to use amqp but threw it away for 0mq
[Tuesday 21 September 2010] [14:41:40] <cremes> i do some pub/sub and some request/reply style message passing
[Tuesday 21 September 2010] [14:41:55] <cremes> that’s the short version
[Tuesday 21 September 2010] [14:42:01] <AltraMayor> How do your components deal with failure? Do you just restart them?
[Tuesday 21 September 2010] [14:42:27] <cremes> it depends
[Tuesday 21 September 2010] [14:42:45] <cremes> if certain components fail, it is fatal for the whole system
[Tuesday 21 September 2010] [14:42:58] <AltraMayor> By the way, PNUTS is a NoSQL that supports replication/accesses/updates among datacenters.
[Tuesday 21 September 2010] [14:43:05] <cremes> if others fail, i detect their absence and just restart/resend their jobs to surviving nodes
[Tuesday 21 September 2010] [14:43:32] <cremes> then perhaps you want to look at mongodb as something similar; 0mq has nothing to do with nosql
[Tuesday 21 September 2010] [14:43:40] <cremes> it’s not a database in any way, shape or form
[Tuesday 21 September 2010] [14:47:47] <AltraMayor> I see that 0mq isn’t a database, but messaging brokers are key in PNUTS’ design. Mongodb is the best NoSQL that I know so far, but it don’t support datacenters as graceful as PNUTS. And, I’m considering implement some pieces, I just didn’t want to start from so low as the message broker.
[Tuesday 21 September 2010] [14:48:36] <guido_g> then get something w/ a broker
[Tuesday 21 September 2010] [14:49:36] <AltraMayor> Ok.
[Tuesday 21 September 2010] [14:50:08] <AltraMayor> Thank you for your time!
[Tuesday 21 September 2010] [14:50:39] Quit AltraMayor has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [14:50:52] <guido_g> best would be broker which stores the messages in a reliable way distributed among datacenters
[Tuesday 21 September 2010] [14:51:28] <guido_g> now wating for the stack overflow...
[Tuesday 21 September 2010] [15:30:36] Quit cpscotti has left this server (Ping timeout: 252 seconds).
[Tuesday 21 September 2010] [15:49:18] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [16:25:49] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [16:56:11] <Samy> like the new website
[Tuesday 21 September 2010] [16:56:49] <Samy> It would be nice if there was a more obvious link to the source repository.
[Tuesday 21 September 2010] [16:59:06] <Samy> Ah, zeromq.com != zeromq.org
[Tuesday 21 September 2010] [17:07:55] <mato> Samy: yeah, the .com site is intended as a portal for new users
[Tuesday 21 September 2010] [17:08:09] <mato> Samy: those who just want a first quick look
[Tuesday 21 September 2010] [17:11:21] <jond> mato: hi
[Tuesday 21 September 2010] [17:14:31] <mato> jond: hi there
[Tuesday 21 September 2010] [17:15:04] <jond> mato: i’ve been looking at the select.cpp and i think there is an issue
[Tuesday 21 September 2010] [17:15:23] <jond> because I’m fixing something similar in the dayjob
[Tuesday 21 September 2010] [17:17:08] <mato> jond: what kind of issue?
[Tuesday 21 September 2010] [17:17:14] <jond> as well as checking that the array doesnt get to FD_SETSIZE, it needs to check that the
fd_ in add_fd is actually greater than FD_SETSIZE
[Tuesday 21 September 2010] [17:18:45] <mato> why would the actual fd matter?
[Tuesday 21 September 2010] [17:18:49] <jond> FD_SET, CLR etc don’t work on linux above FD_SETSIZE. it’s on the man page at the bottom; I had a process with 6000 fd’s open and once the sockets started connecting and getting fd > 1024 bad things happen
[Tuesday 21 September 2010] [17:19:31] <jond> fd_set is normally implemented as a bitvector of fixed size
[Tuesday 21 September 2010] [17:19:33] <mato> ah, you’re right
[Tuesday 21 September 2010] [17:19:35] <mato> yeah
[Tuesday 21 September 2010] [17:19:46] <mato> good point
[Tuesday 21 September 2010] [17:20:21] <mato> well, the select implementation has not really been stress-tested properly
[Tuesday 21 September 2010] [17:20:40] <mato> it’s only there for windows benefit (and vms I guess)
[Tuesday 21 September 2010] [17:20:52] <jond> we raised call with redhat and even if you recompile glibc etc the kernel has it at 1024. My problem is that the 6000 fd’s are files opened early and the sockets connect later and get to high fd.
[Tuesday 21 September 2010] [17:21:02] <jond> i’m changing to poll
[Tuesday 21 September 2010] [17:21:27] <mato> well, if you’re dealing with 000’s of fds you really should not be using select at all :-)
[Tuesday 21 September 2010] [17:21:28] <jond> on solaris I believe you can just redefine FD_SETSIZE
[Tuesday 21 September 2010] [17:21:48] <mato> and poll is not too performant with lots of fds either, you really want an edge-triggered interface
[Tuesday 21 September 2010] [17:22:27] <mato> anyhow, thanks for the tip; i’ll make a note to make a patch that adds in the appropriate assertions
[Tuesday 21 September 2010] [17:23:03] <jond> yep I know, but I ‘inherited’ this code; we only have a 600 tcp connections, but open lots of files....
[Tuesday 21 September 2010] [17:24:38] <jond> mato: edge-triggered, you mean epoll?
[Tuesday 21 September 2010] [17:27:02] <mato> jond: yes
[Tuesday 21 September 2010] [17:27:21] <mato> jond: unfortunately epoll never got standardised so is available only on Linux
[Tuesday 21 September 2010] [17:27:42] <mato> jond:
*BSD has kqueue, Solaris has Event Completion
[Tuesday 21 September 2010] [17:27:53] <jond> i’ll take a look tommorrow; thanks for the tip;
[Tuesday 21 September 2010] [17:28:09] <jond> mato: cheers
[Tuesday 21 September 2010] [17:36:42] <bgranger> mato: did you see my reply about the message related deadlock you were seeing?
[Tuesday 21 September 2010] [17:38:31] <jond> bgranger: i did. is there a link to that pyzmq branch
[Tuesday 21 September 2010] [17:38:39] <bgranger> Sure, right here...
[Tuesday 21 September 2010] [17:38:54] <dermoth> Is there any example of XREQ/XREP out there? I’ve used REP/REP successfully in the past and now I’d like to add devices in the middle, and I believe I need XREQ/XREP for that.
[Tuesday 21 September 2010] [17:39:11] <bgranger> Here is the class
[Tuesday 21 September 2010] [17:40:03] <bgranger> We are still doing code review so a few more things might change in the API, but I think things are mostly in place.
[Tuesday 21 September 2010] [17:40:49] <mato> bgranger: hi
[Tuesday 21 September 2010] [17:41:04] <bgranger> hi
[Tuesday 21 September 2010] [17:41:12] <mato> bgranger: yes, i saw that, am travelling most of this week (Linux Kongress in Nurnberg, Martin Sustrik is presenting)
[Tuesday 21 September 2010] [17:41:21] <bgranger> Nice
[Tuesday 21 September 2010] [17:41:30] <jond> bgranger: thanks, i’ll take a look. I think that function that’s added in py2.7 for PendingCall looks interesting and could still provide a version which doesnt need the MesssageTrcaker
[Tuesday 21 September 2010] [17:41:33] <mato> bgranger: will look at it next week once I get back home
[Tuesday 21 September 2010] [17:41:48] <bgranger> Great, sounds good, by then it will probably be in our master.
[Tuesday 21 September 2010] [17:41:50] <mato> bgranger: can’t really test it reliably while on the road.
[Tuesday 21 September 2010] [17:41:59] <bgranger> OK
[Tuesday 21 September 2010] [17:53:17] Quit mytrile has left this server (Read error: Connection reset by peer).
*** on Tue Sep 21 17:58:29 2010
[Tuesday 21 September 2010] [17:58:29] Topic The channel topic is “Welcome!”.
[Tuesday 21 September 2010] [17:58:33] Mode Channel modes: no messages from outside, topic protection
[Tuesday 21 September 2010] [17:58:33] Created This channel was created on 2010-02-05 10:44.
[Tuesday 21 September 2010] [17:58:43] <dermoth> [bbl]
[Tuesday 21 September 2010] [18:44:02] Quit jhawk28 has left this server (Ping timeout: 265 seconds).
[Tuesday 21 September 2010] [18:51:17] Quit mytrile has left this server (Ping timeout: 265 seconds).
[Tuesday 21 September 2010] [19:27:45] <tylergillies> whats the difference between req/rep and upstream/downstream?
[Tuesday 21 September 2010] [19:31:11] <vanadium> req/rep are bidirectional/back and forth, upstream/downstream are push/pull and one of them always sends and the other always receives
[Tuesday 21 September 2010] [19:38:36] Quit dbudworth has left this server (Quit: dbudworth).
[Tuesday 21 September 2010] [19:46:19] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 21 September 2010] [20:27:15] Part starkdg has left this channel.
[Tuesday 21 September 2010] [20:32:04] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [20:43:19] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [20:59:19] Quit
jhawk28_ has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [21:03:26] <dermoth> actually one of the issues that I was seeing is obvious... in a php worker I want to have a persistent socket that use req/rep (or the x equivalent if needed), but if the backend isn’T responding timely I just want to leave early and go on, but then when the next request comes in the socket is still expecting a reply... how can I work around this?
[Tuesday 21 September 2010] [21:03:41] <dermoth> i guess that’S a good one for the ML :)
[Tuesday 21 September 2010] [21:05:59] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 21 September 2010] [21:18:31] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Tuesday 21 September 2010] [22:10:36] Quit jhawk28 has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [00:01:14] Quit RobertLJ has left this server (Ping timeout: 264 seconds).
[Wednesday 22 September 2010] [00:19:54] Quit kleppari has left this server (Ping timeout: 272 seconds).
*** on Wed Sep 22 07:12:46 2010
[Wednesday 22 September 2010] [07:12:46] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [07:12:50] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [07:12:50] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [07:53:53] <raz> hm
[Wednesday 22 September 2010] [07:53:58] <raz> looking at the divide&conquer example i wonder
[Wednesday 22 September 2010] [07:54:18] <raz> how does zmq behave when a worker does not indicate it finished? (e.g. because it crashed)
[Wednesday 22 September 2010] [07:54:25] <raz> will the task be re-queued to a different worker?
[Wednesday 22 September 2010] [08:00:10] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Wednesday 22 September 2010] [08:00:10] Notice -NickServ- You are now identified for travlr.
[Wednesday 22 September 2010] [08:00:10] CTCP Received Version request from frigg.
[Wednesday 22 September 2010] [08:00:14] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [08:00:18] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [08:00:18] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [08:02:12] <guido_g> no, you have to do it yourself
[Wednesday 22 September 2010] [08:07:53] <raz> hm
[Wednesday 22 September 2010] [08:08:36] <raz> the taskvent example doesn’t seem to inspect the return value at all
[Wednesday 22 September 2010] [08:08:53] <raz> would s_send return an error if no worker picks the message up?
[Wednesday 22 September 2010] [08:09:27] * raz tries to find some semantic overview of the different modes/types
[Wednesday 22 September 2010] [08:10:37] <raz> the zmq_send man-page does not mention such an error tho
[Wednesday 22 September 2010] [08:11:19] <guido_g> there should be a description what happens for the socket types if something goes wrong
[Wednesday 22 September 2010] [08:11:51] * guido_g needs an new english grammar module though
[Wednesday 22 September 2010] [08:12:20] <raz> hmyea
[Wednesday 22 September 2010] [08:12:30] <raz> the ZMQ_PUSH section doesn’t tell me what happens when a msg can not be delivered tho
[Wednesday 22 September 2010] [08:12:41] <raz> or rather.. i assumed there was some sort of acknowledgement going on
[Wednesday 22 September 2010] [08:12:45] <guido_g> nothing
[Wednesday 22 September 2010] [08:12:46] <raz> which is apparently on the case
[Wednesday 22 September 2010] [08:13:19] <raz> s/on/not/
[Wednesday 22 September 2010] [08:14:02] <guido_g> there is no guarantee for delivery in mq
[Wednesday 22 September 2010] [08:15:03] <raz> hmm
[Wednesday 22 September 2010] [08:30:56] Quit xla has left this server (Quit: leaving).
[Wednesday 22 September 2010] [08:37:26] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 22 September 2010] [08:37:48] <ajsie> zeromq doesnt support persistence?
[Wednesday 22 September 2010] [08:50:47] Quit xla has left this server (Quit: leaving).
[Wednesday 22 September 2010] [08:57:32] Quit sbahra has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [09:15:01] <guido_g> it can store messages, but that’s not what you know from broker based solutions
[Wednesday 22 September 2010] [09:23:58] Quit zsxxsz has left this server (Quit: Page closed).
[Wednesday 22 September 2010] [09:28:49] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 22 September 2010] [09:32:55] <keffo> mm, nice.. code dated 3/11/78
[Wednesday 22 September 2010] [09:37:24] <vanadium> If I have one push sockets sending workloads to a number of threads with pull sockets, the work will be evenly distributed over all threads even if some of them take way longer per workload, right?
[Wednesday 22 September 2010] [09:37:38] <vanadium> It is just a round-robin thing and not actually communicating back who is ready for more work or anything?
[Wednesday 22 September 2010] [09:50:11] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 22 September 2010] [09:50:42] <drbobbeaty> vanadium: I’ve not used the PUSH/PULL sockets before, but I believe that you are correct in your assumption. You have a single socket (PUSH) that’s bound to some URL, and a series of “worker threads” that each has a socket (PULL) that’s connected to that same URL. Then, as the single “producer” socket sends out messages, they are picked up and worked on, in turn, by the worker threads.
[Wednesday 22 September 2010] [09:51:24] <drbobbeaty> As i understand it, there are several ways to handle the distribution - round-robin, least-recently-used, etc., but I’m not as familiar with those concepts or how to set them up.
[Wednesday 22 September 2010] [09:51:24] <vanadium> Thank you
[Wednesday 22 September 2010] [09:51:58] <drbobbeaty> no problem
[Wednesday 22 September 2010] [09:58:09] Part ajsie has left this channel.
[Wednesday 22 September 2010] [10:03:28] <guido_g> re
[Wednesday 22 September 2010] [10:13:17] <raz> has anyone used zmq with ruby eventmachine?
[Wednesday 22 September 2010] [10:13:20] <raz> does it blend?
[Wednesday 22 September 2010] [10:17:39] <cremes> raz: the 2.0.x branch doesn’t blend well with EM; the 2.1.x branch might
[Wednesday 22 September 2010] [10:18:10] <raz> well, i have an existing em app that i’d like to add zmq to
[Wednesday 22 September 2010] [10:18:30] <raz> zmqmachine looks like it’s meant to be a replacement for em
[Wednesday 22 September 2010] [10:18:41] <cremes> right
[Wednesday 22 September 2010] [10:19:01] <cremes> you’d have to wire the 0mq sockets into the EM reactor loop
[Wednesday 22 September 2010] [10:19:09] <cremes> that isn’t possible with any 0mq release right now
[Wednesday 22 September 2010] [10:19:25] <raz> hrm
[Wednesday 22 September 2010] [10:19:28] <cremes> the 2.1.x branch has a few new file descriptor elements exposed which will make it feasible to do so
[Wednesday 22 September 2010] [10:19:43] <raz> any dirty trick possible with a thread or so?
[Wednesday 22 September 2010] [10:19:55] <cremes> of course, but that’s cheating :)
[Wednesday 22 September 2010] [10:19:59] <raz> hehe
[Wednesday 22 September 2010] [10:20:11] <raz> i don’t need it super pretty or fast, but super reliable would be nice ;)
[Wednesday 22 September 2010] [10:21:05] <raz> sadly my eventmachine fu isn’t all that advanced, not sure if i’d get the dispatching between eventmachine and a separate thread right
[Wednesday 22 September 2010] [10:21:06] <cremes> you would need to run the 0mq stuff in a thread separate from EM and any interaction would need to be scheduled via EM.schedule
[Wednesday 22 September 2010] [10:21:23] <cremes> 0mq -> EM would be via EM.schedule
[Wednesday 22 September 2010] [10:21:33] <cremes> EM -> 0mq could be handled straight up
[Wednesday 22 September 2010] [10:22:04] <raz> except when the 0mq call blocks (in the latter case)
[Wednesday 22 September 2010] [10:22:16] <raz> guess i should use a queue both ways
[Wednesday 22 September 2010] [10:22:23] <cremes> raz: you’d have to pass ZM_NOBLOCK to all send/recv calls
[Wednesday 22 September 2010] [10:22:48] <raz> hmm
[Wednesday 22 September 2010] [10:23:15] <cremes> you could maybe run zmqmachine and EM in parallel; all of that NO_BLOCK stuff is handled for you then
[Wednesday 22 September 2010] [10:23:18] <raz> sounds like that may be worth a shot
[Wednesday 22 September 2010] [10:23:25] <cremes> let us know how it turns out
[Wednesday 22 September 2010] [11:50:05] Quit dbudworth has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [11:53:39] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 22 September 2010] [12:48:09] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [12:54:38] Quit plol has left this server (Ping timeout: 264 seconds).
[Wednesday 22 September 2010] [12:57:31] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 22 September 2010] [13:35:07] Part dbudworth has left this channel.
[Wednesday 22 September 2010] [13:35:39] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 22 September 2010] [13:36:58] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [13:45:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [13:45:56] Quit plq has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [13:49:42] <mankins> hi–i’ve got an installation problem with 2.0.9 under linux. configure: error: cannot link with -luuid, install uuid-dev.
[Wednesday 22 September 2010] [13:49:58] <mankins> only I’ve already got uuid-devel installed (under fedora core 8 on ec2)
[Wednesday 22 September 2010] [13:50:39] <mankins> libuuid.so.1 is at /lib ... not sure where to go next. Does this ring any bells?
[Wednesday 22 September 2010] [13:54:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [13:56:58] * mankins looks to mirror: oh! ln -s /usr/lib/libuuid.so.1 /usr/lib/libuuid.so fixed it.
[Wednesday 22 September 2010] [14:14:59] * mankins oh, maybe e2fsprogs-devel as well.
[Wednesday 22 September 2010] [14:18:21] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [14:34:10] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [14:39:22] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [14:44:52] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [15:02:18] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 22 September 2010] [16:05:41] Quit xla has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [16:08:45] Quit
antares_ has left this server (Quit:
antares_).
[Wednesday 22 September 2010] [16:29:17] <CIA-20> jzmq: 03Stefan Majer 07master * rf3aa751 10/ (jzmq.spec pom.xml): RPM Packaging and Mavenized, first step. -
http://bit.ly/a6fxhX[Wednesday 22 September 2010] [17:07:10] <cremes> i’m wondering if anyone has seen an assertion failure in 2.0.9
[Wednesday 22 September 2010] [17:07:11] <cremes> Assertion failed: !engine (session.cpp:287)
[Wednesday 22 September 2010] [17:07:49] <cremes> i’m hitting that a lot but i can’t figure out why; i’m having a hard time reducing the code to a small example
[Wednesday 22 September 2010] [17:11:54] <pieterh> cremes: that’s a reported issue afair
[Wednesday 22 September 2010] [17:12:11] <pieterh> it’s caused by... /me vaguely recalls... identity crisis
[Wednesday 22 September 2010] [17:12:23] * pieterh goes to check issue tracker
[Wednesday 22 September 2010] [17:12:27] <cremes> ah... 2+ sockets with same identity...?
[Wednesday 22 September 2010] [17:12:37] <pieterh> yeah
[Wednesday 22 September 2010] [17:12:45] <pieterh> evil twin syndrome
[Wednesday 22 September 2010] [17:12:47] <cremes> cool, then i know where my bug is.... :)
[Wednesday 22 September 2010] [17:13:00] <pieterh> issue 30
[Wednesday 22 September 2010] [17:13:23] <pieterh> hey chuck, Chicago rocks :-)
[Wednesday 22 September 2010] [17:13:59] <cremes> i’m glad you had a good time... may i assume the trip was worthwhile from both a pleasure and business perspective?
[Wednesday 22 September 2010] [17:14:43] * pieterh is typing with his 1-month son in his arms... slightly tricky
[Wednesday 22 September 2010] [17:14:49] <cremes> heh
[Wednesday 22 September 2010] [17:15:07] <pieterh> 0mq + chicago = boom!
[Wednesday 22 September 2010] [17:15:21] <cremes> good boom or bad boom? ;)
[Wednesday 22 September 2010] [17:15:32] <pieterh> the good kind :-)
[Wednesday 22 September 2010] [17:15:50] <cremes> great... next time you come out bring the team
[Wednesday 22 September 2010] [17:16:24] <cremes> oops, gotta run; catch ya later!
[Wednesday 22 September 2010] [17:17:32] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 22 September 2010] [17:17:56] <pieterh> cyl
[Wednesday 22 September 2010] [17:21:10] <mato> pieterh: yo
[Wednesday 22 September 2010] [17:21:18] <mato> pieterh: just got back from dinner, etc with sustrik
[Wednesday 22 September 2010] [17:21:38] <mato> pieterh: linux-kongress starts tomorrow, martin is speaking around 5pm
[Wednesday 22 September 2010] [17:24:00] <pieterh> hey mato :-)
[Wednesday 22 September 2010] [17:24:11] <pieterh> how’s nurnberg?
[Wednesday 22 September 2010] [17:24:21] <mato> nice
[Wednesday 22 September 2010] [17:24:27] <mato> good beer
[Wednesday 22 September 2010] [17:24:35] <mato> some kind of local beer fest is on
[Wednesday 22 September 2010] [17:24:39] <pieterh> in germany? lol...
[Wednesday 22 September 2010] [17:25:20] <pieterh> there’s some kind of wine festival in bratislava next weekend
[Wednesday 22 September 2010] [17:25:49] <pieterh> hey, mato, random question about messages
[Wednesday 22 September 2010] [17:26:08] <pieterh> i vaguely recall El Sustrik saying that zmq_send nullifies the message it sends
[Wednesday 22 September 2010] [17:26:24] <pieterh> even insisting this should be clearly explained in the guide
[Wednesday 22 September 2010] [17:26:33] <mato> zmq_recv does
[Wednesday 22 September 2010] [17:26:39] <mato> send doesn’t afaik
[Wednesday 22 September 2010] [17:26:40] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 22 September 2010] [17:26:47] <pieterh> ? recv nullifies the message?
[Wednesday 22 September 2010] [17:26:56] * pieterh needs another glass of vino to understand this
[Wednesday 22 September 2010] [17:27:15] <pieterh> ah, before receiving, previous contents are discarded...
[Wednesday 22 September 2010] [17:28:25] <pieterh> mato: nah, it was send
[Wednesday 22 September 2010] [17:28:32] <pieterh> i found the line in the userguide:
[Wednesday 22 September 2010] [17:28:33] <pieterh> Note than when you have passed a message to zmq_send[3], 0MQ will clear the message, i.e. set the data to empty. You cannot send the same message twice, and you cannot access the message data after sending it.
[Wednesday 22 September 2010] [17:28:44] <pieterh> yet this is wrong
[Wednesday 22 September 2010] [17:28:53] Quit plol has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [17:29:15] <pieterh> could you bounce this off Sustrik?
[Wednesday 22 September 2010] [17:30:04] <mato> that’s wrong
[Wednesday 22 September 2010] [17:30:12] <mato> of course you can send a message multiple times
[Wednesday 22 September 2010] [17:30:15] <mato> i do it all the time
[Wednesday 22 September 2010] [17:30:18] <starkdg> i thought zmq_msg_close() is what kills messages ?
[Wednesday 22 September 2010] [17:30:24] <mato> yes, precisely
[Wednesday 22 September 2010] [17:30:42] <mato> recv is special in that it destroys the content of the message being received into (if any)
[Wednesday 22 September 2010] [17:30:46] <mato> which makes sense
[Wednesday 22 September 2010] [17:30:50] <mato> send does not touch it
[Wednesday 22 September 2010] [17:30:51] <pieterh> mato: that all makes sense
[Wednesday 22 September 2010] [17:31:13] <pieterh> yet I wrote that after sustrik explicitly asked me to emphasize that
[Wednesday 22 September 2010] [17:31:20] <pieterh> maybe he meant recv not send...
[Wednesday 22 September 2010] [17:31:25] <pieterh> or i noted it wrong
[Wednesday 22 September 2010] [17:31:32] <mato> probably
[Wednesday 22 September 2010] [17:31:47] <pieterh> send is 100% safe?
[Wednesday 22 September 2010] [17:31:57] <mato> it’s quite obviously wrong, i have lots of code that sends the same msg_t X times
[Wednesday 22 September 2010] [17:32:00] <pieterh> kurwa
[Wednesday 22 September 2010] [17:32:03] <pieterh> ok, thanks...
[Wednesday 22 September 2010] [17:32:04] <mato> :-)
[Wednesday 22 September 2010] [17:32:58] <starkdg> you can recieve and send the same message, right ? ive been doing that
[Wednesday 22 September 2010] [17:33:49] <mato> yes
[Wednesday 22 September 2010] [17:58:24] Quit mankins has left this server (Quit: mankins).
[Wednesday 22 September 2010] [18:01:59] Quit ModusPwnens has left this server.
*** on Wed Sep 22 18:31:08 2010
[Wednesday 22 September 2010] [18:31:08] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [18:31:12] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [18:31:12] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [18:31:28] Quit travlr has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [18:47:50] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 22 September 2010] [18:50:43] Quit starkdg has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [18:58:33] Part jashmenn has left this channel.
[Wednesday 22 September 2010] [19:01:12] Nick MJW is now known as mjw9100.
[Wednesday 22 September 2010] [19:08:43] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [19:18:53] Quit starkdg has left this server (Ping timeout: 245 seconds).
[Wednesday 22 September 2010] [19:21:12] Part mjw9100 has left this channel.
[Wednesday 22 September 2010] [19:24:04] Part mjw9100 has left this channel.
[Wednesday 22 September 2010] [19:25:33] Quit friendshi has left this server (Quit: friendshi).
[Wednesday 22 September 2010] [19:42:25] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [19:42:38] Quit pieterh has left this server (Ping timeout: 245 seconds).
[Wednesday 22 September 2010] [19:54:03] Part mikejs has left this channel.
[Wednesday 22 September 2010] [20:56:45] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Wednesday 22 September 2010] [21:03:03] Quit andrewvc has left this server (Quit: andrewvc).
[Wednesday 22 September 2010] [21:59:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [22:40:39] Quit friendshi has left this server (Client Quit).
[Wednesday 22 September 2010] [22:45:48] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 22 September 2010] [23:03:12] Quit grahamalot has left this server (Quit: grahamalot).
[Wednesday 22 September 2010] [23:46:30] Quit yrashk has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [00:28:54] <Pe_Ell> anyone know if there is a roadmap for release dates? Just wondering since I didn’t notice one on github or the site off hand...
[Thursday 23 September 2010] [01:11:17] Quit BooTheHamster1 has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:11:37] Join BooTheHamster1 has joined this channel (
~boo@94.25.51.179).
[Thursday 23 September 2010] [01:24:51] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:27:37] Quit BooTheHamster1 has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:39:53] <guido_g> nope, there is no such thing
[Thursday 23 September 2010] [01:40:21] <guido_g> standard quote is “it’ll be released when its ready” :)
[Thursday 23 September 2010] [01:42:47] <Pe_Ell> heh, k
[Thursday 23 September 2010] [01:43:23] <Pe_Ell> there’s a feature in 2.1 that I’m waiting for... wasn’t sure how stable the git branch was... guess I’ll just keep on waiting... :)
[Thursday 23 September 2010] [01:43:37] <guido_g> if you’re around longer, ask pieterh, mato or sustrik, they do know the details
[Thursday 23 September 2010] [01:44:20] <Pe_Ell> k. I’ll just leave this open for now while I read more code/docs...
[Thursday 23 September 2010] [01:55:59] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [01:56:47] Quit plol has left this server (Ping timeout: 240 seconds).
[Thursday 23 September 2010] [01:59:54] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [02:13:01] Quit raz has left this server (Quit: fump).
[Thursday 23 September 2010] [02:58:15] Quit andrewvc has left this server (Quit: andrewvc).
[Thursday 23 September 2010] [03:19:36] Quit plol has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [03:42:56] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [04:36:10] Nick
mikko__ is now known as mikko.
[Thursday 23 September 2010] [05:00:00] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 23 September 2010] [05:25:10] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 23 September 2010] [05:49:09] Quit jugg has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [06:42:11] Join plq has joined this channel (
~plq@78.189.30.218).
[Thursday 23 September 2010] [08:25:06] Part mjw9100 has left this channel.
[Thursday 23 September 2010] [08:53:57] <pieter_hintjens> hi folk, it’s quiet today
[Thursday 23 September 2010] [08:54:21] <pieter_hintjens>
*folks
[Thursday 23 September 2010] [08:54:32] <pieter_hintjens> I’ve pushed out a new version of the user guide
[Thursday 23 September 2010] [09:01:50] Quit mytrile has left this server (Ping timeout: 252 seconds).
[Thursday 23 September 2010] [09:28:17] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [09:55:49] Quit sbahra has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [09:56:49] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [10:01:27] Quit plol has left this server (Read error: Operation timed out).
[Thursday 23 September 2010] [10:18:19] Quit plq has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [10:39:02] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Thursday 23 September 2010] [10:58:13] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [11:19:34] <friendshi> hi, I’m having an issue when trying to do non-block recv... I always immediately receive an exception ‘Resource temporarily unavailable’, anyone know where i might be going wrong?
[Thursday 23 September 2010] [11:20:30] <guido_g> maybe the connect wasn’t done at this point in time
[Thursday 23 September 2010] [11:20:52] <friendshi> hmm it works fine with i do blocking, and i have tried sleeping for 5+ seconds between connect and recv
[Thursday 23 September 2010] [11:21:13] <guido_g> because the real connection handling is in a seperate thread, you might do the receive while the underlying connection is not yet established
[Thursday 23 September 2010] [11:23:33] <friendshi> i dont think that can be it... when i do it without block it begins reading my messages immediately... unless i am misunderstanding what you are suggesting
[Thursday 23 September 2010] [11:23:42] <friendshi>
*without nonblock
[Thursday 23 September 2010] [11:24:04] <guido_g> i didn’t suggest something
[Thursday 23 September 2010] [11:24:19] <friendshi> sorry i just mean, what you are saying the issue could be
[Thursday 23 September 2010] [11:24:40] <guido_g> one idea is to make the first receive blocking and all others not
[Thursday 23 September 2010] [11:25:01] <guido_g> if this works, it’s time for an issue i’d say
[Thursday 23 September 2010] [11:26:26] * guido_g checks the docu
[Thursday 23 September 2010] [11:26:27] <friendshi> same issue :(
[Thursday 23 September 2010] [11:26:45] <friendshi> it does receive first message fine, then immediate fail on nonblock recv
[Thursday 23 September 2010] [11:26:53] <guido_g> then try to make a small program that shows the problem and show the code
[Thursday 23 September 2010] [11:27:08] <friendshi> sure
[Thursday 23 September 2010] [11:45:32] Nick rbancrof1 is now known as rbancroft.
[Thursday 23 September 2010] [11:45:52] <vagn> greetings. Is there an example somewhere on how to shutdown a ZMQ_REP socket so the otherside doesn’t hang when the service returns?
[Thursday 23 September 2010] [11:46:30] Join ChicagoBud has joined this channel (
~bud@208.78.39.48).
[Thursday 23 September 2010] [11:46:57] <vagn> I want to stop getting requests, but finish all the replies, then exit the program.
[Thursday 23 September 2010] [11:47:48] <vagn> but the requests are queued, right? so the other side things it should get a reply. so basically I have to shutdown the incoming queue, too.
[Thursday 23 September 2010] [11:47:57] <vagn>
*thinks
[Thursday 23 September 2010] [11:53:19] <guido_g> i’d do this kind of synchronization with explicit messages, much easier imho
[Thursday 23 September 2010] [11:54:45] <vagn> thinking some more... that still doesn’t cure tha case where the server crashes.
[Thursday 23 September 2010] [11:56:02] <vagn> maybe I should start over and do things with pub/sub or multicast. I don’t care if I lose a message. the next one that comes along is good enough.
[Thursday 23 September 2010] [12:14:43] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 23 September 2010] [12:14:56] Quit
kayiwa_ has left this server (Quit:
kayiwa_).
[Thursday 23 September 2010] [12:16:33] <friendshi> i find that my problem was that in pyzmq document/source it states ‘If NOBLOCK is set, this method will return None if a message is not ready’ but that is untrue, I believe it is throwing exception is no message exist
[Thursday 23 September 2010] [12:16:55] <friendshi> maybe i am misreading or misunderstanding though
[Thursday 23 September 2010] [12:22:21] <keffo> oh by my might beard! I hate 3g
[Thursday 23 September 2010] [12:37:51] Quit xla has left this server (Quit: leaving).
[Thursday 23 September 2010] [12:58:56] <bgranger> friendshi: hi
[Thursday 23 September 2010] [13:19:51] Quit vagn has left this server (Quit: Ex-Chat).
[Thursday 23 September 2010] [14:56:38] Quit mytrile has left this server (Ping timeout: 272 seconds).
[Thursday 23 September 2010] [15:04:07] Quit miguelito has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [15:39:01] Notice -Martinp23- :[Global Notice] Hi folks. One of our servers (jordan) needs to undergo emergency maintenance in the next 30 minutes. The maintenance window will be 48 hours or so. The result of this will be a netsplit as 3000 users, who have already been notified seperately, disconnect. I’m sorry for the inconvenience. If you’ve any questions, pop into #freenode. Thanks!
[Thursday 23 September 2010] [15:40:44] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [16:00:40] Quit miguelito has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:40] Quit yrashk has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:41] Quit mato has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:41] Quit jond has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:33:01] Quit plol has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [16:42:38] Quit xla has left this server (Ping timeout: 264 seconds).
[Thursday 23 September 2010] [16:44:07] Quit halfie has left this server (Quit: Leaving).
[Thursday 23 September 2010] [17:01:17] Quit swapi has left this server (Quit: Page closed).
[Thursday 23 September 2010] [17:13:05] Quit bgranger has left this server (Read error: Operation timed out).
[Thursday 23 September 2010] [17:22:48] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 23 September 2010] [17:23:03] Quit miguelito has left this server (Ping timeout: 245 seconds).
[Thursday 23 September 2010] [18:01:00] Quit andrewvc has left this server (Quit: andrewvc).
[Thursday 23 September 2010] [18:15:57] Quit friendshi has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [19:35:43] Part vanadium has left this channel.
[Thursday 23 September 2010] [19:37:44] Quit
mw_ has left this server (Quit:
mw_).
[Thursday 23 September 2010] [19:39:25] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Thursday 23 September 2010] [19:47:22] Quit
mw_ has left this server (Quit:
mw_).
[Thursday 23 September 2010] [20:43:27] Quit
miguelito_ has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [20:49:45] Quit iFire has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [20:55:26] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [21:21:38] <jhawk28> does any other protocols than tcp work on Windows?
[Thursday 23 September 2010] [21:50:58] Quit xla has left this server (Ping timeout: 245 seconds).
[Thursday 23 September 2010] [22:28:06] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [22:31:23] Quit jhawk28 has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [22:32:57] Part starkdg has left this channel.
[Thursday 23 September 2010] [23:55:51] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [00:00:08] Quit mjw9100 has left this server (Remote host closed the connection).
[Friday 24 September 2010] [01:38:56] Quit jsimmons has left this server (Quit: Leaving).
[Friday 24 September 2010] [02:24:03] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [02:27:48] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [02:28:13] Quit guido_g has left this server (Quit: Boom...).
[Friday 24 September 2010] [02:33:22] Quit pieter_hintjens has left this server (Ping timeout: 255 seconds).
[Friday 24 September 2010] [03:09:06] Nick Nickname already in use. Trying
travlr__.
[Friday 24 September 2010] [03:09:11] Topic The channel topic is “Welcome!”.
[Friday 24 September 2010] [03:09:12] CTCP Received Version request from frigg.
[Friday 24 September 2010] [03:09:12] Notice -NickServ- travlr__ is not a registered nickname.
[Friday 24 September 2010] [03:09:15] Mode Channel modes: no messages from outside, topic protection
[Friday 24 September 2010] [03:09:15] Created This channel was created on 2010-02-05 10:44.
[Friday 24 September 2010] [03:09:43] Quit
travlr_ has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [03:10:14] Quit Pe_Ell has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [03:19:13] Quit SecretAgent has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [03:20:26] Quit AlexB has left this server (Ping timeout: 264 seconds).
[Friday 24 September 2010] [03:20:40] Join AlexB has joined this channel (
alex@brasetvik.com).
[Friday 24 September 2010] [03:33:43] Quit Pe-Ell has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [03:34:50] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [04:08:28] Quit xla has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [04:59:18] Quit drbobbeaty has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [05:02:02] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [05:07:43] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 24 September 2010] [05:25:26] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 24 September 2010] [05:30:46] Quit guido_g has left this server (Quit: Boom...).
[Friday 24 September 2010] [05:34:07] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [07:27:21] <Zao> Assuming you manage to build it, I can’t see why OpenPGM wouldn’t work.
[Friday 24 September 2010] [07:28:31] Quit keffo has left this server (Quit: Leaving).
[Friday 24 September 2010] [07:32:40] Part
i__ has left this channel.
[Friday 24 September 2010] [07:40:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [08:48:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [08:55:50] Quit plol has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [09:07:09] Part mjw9100 has left this channel.
[Friday 24 September 2010] [09:29:53] Part mjw9100 has left this channel.
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r7073557 10/ src/org/zeromq/ZMQ.java : Javadoc, reduction of the visibility of a lot of flags reduced to private because they are only used inside public setters/getters. Fix of some checkstyle warnings. -
http://bit.ly/aGlNur[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r34d656f 10/ src/org/zeromq/ZMQ.java :
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Deprecation of ZMQ.UPSTREAM and ZMQ.DOWNSTREAM
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Socket.subscribe and Socket.unsubscribe only on a Socket of Type ZMQ.SUB -
http://bit.ly/cQRpmM[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r86512f8 10/ src/org/zeromq/ZMQ.java :
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Revert the subscribe/unsubscribe check,
[Friday 24 September 2010] [09:45:34] <CIA-20> jzmq: Whitespace and identation with zeromq rules -
http://bit.ly/9wrDyQ[Friday 24 September 2010] [09:45:35] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * r3ba6cc1 10/ src/org/zeromq/ZMQ.java : Pulled changes from Stefan Majer to improve documentation and visibility. Made minor modifications. -
http://bit.ly/awIGSC[Friday 24 September 2010] [09:55:50] Part mjw9100 has left this channel.
[Friday 24 September 2010] [10:53:29] <codebeaker> hey, quick sanity check - can I (should I?) include the zeromq repository in my own project as a git submodule ?
[Friday 24 September 2010] [11:05:00] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [11:10:43] Quit rhi has left this server (Client Quit).
[Friday 24 September 2010] [11:15:43] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [11:16:58] <cremes> codebeaker: that’s a good question for some git experts... i don’ t think there are any in this channel
[Friday 24 September 2010] [11:18:04] <codebeaker> cremes: I don’t need a git expert, my question was more about whether it makes sense, if you distribute a closed source application, between a couple of devlopers (mac and windows targets) how are other people typically including the mq sources in their projects
[Friday 24 September 2010] [11:18:46] <codebeaker> I noticed there were separate downloads available for Windows and Unix, so that would lead me to believe you have to build them differently
[Friday 24 September 2010] [11:20:51] <cremes> i don’t know how to answer your question... hopefully someone else will come along who can
[Friday 24 September 2010] [11:21:09] <codebeaker> thanks cremes - appreciate the attention anyway
[Friday 24 September 2010] [11:21:33] <codebeaker> basically, I could include the whole repository from mq in my own, and then specify build flags - but I’m not sure how smart that would be
[Friday 24 September 2010] [11:31:29] <pieter_hintjens> codebeaker: including 0MQ as a git submodule would be a pretty good solution, yes
[Friday 24 September 2010] [11:32:09] <pieter_hintjens> at least until 0MQ becomes a standard package on the various distros
[Friday 24 September 2010] [11:32:29] <pieter_hintjens> and definitely a good idea if you want specific versions or branches of 0MQ
[Friday 24 September 2010] [11:33:08] <pieter_hintjens> but the distribution packages are identical except for line endings and compression format
[Friday 24 September 2010] [11:35:49] <codebeaker> pieter_hintjens: ^ thanks that’s exactly the info I needed :-D
[Friday 24 September 2010] [11:35:58] <codebeaker> bew, mq is freakin awesome
[Friday 24 September 2010] [11:36:01] <codebeaker> btw*
[Friday 24 September 2010] [11:36:02] Quit codebeaker has left this server (Quit: codebeaker).
[Friday 24 September 2010] [11:36:04] <pieter_hintjens> codebeaker: hopefully it’s also accurate :-)
[Friday 24 September 2010] [12:00:13] Quit plol has left this server (Ping timeout: 252 seconds).
[Friday 24 September 2010] [12:12:10] <mjw9100> folks, just confirming that an “ordinary” fd for zmq_poll() can be an epoll fd?
[Friday 24 September 2010] [12:19:23] <mjw9100> i.e. can I set zmq_pollitem_t.fd to an epoll() fd?
[Friday 24 September 2010] [12:19:56] <mjw9100> according to the epoll() Q&A, these epoll fd’s are selectable, etc.
[Friday 24 September 2010] [12:23:19] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [12:24:19] <jhawk28> hello
[Friday 24 September 2010] [12:24:59] <jhawk28> having a problem moving the compiled version of the windows libraries to another machine
[Friday 24 September 2010] [12:55:47] <praveenbm5> hello all
[Friday 24 September 2010] [12:56:03] <praveenbm5> i hope i am in the right room to discuss ZeroMQ
[Friday 24 September 2010] [12:56:12] Quit omarkj has left this server (Quit: omarkj).
[Friday 24 September 2010] [12:56:42] <codebeaker> praveenbm5: the name says it all!
[Friday 24 September 2010] [12:56:46] <praveenbm5> is anybody alive here
[Friday 24 September 2010] [12:57:15] <praveenbm5> i was looking on the website regarding some guidance using zmq_poll
[Friday 24 September 2010] [12:57:53] <praveenbm5> i just posted a comment there
[Friday 24 September 2010] [13:03:35] <codebeaker> ^ I’m inqo mq for about 5 minutes now, what’s the behaviour if you broadcast a message to an upstream (is the terminology the right way round there ) - but nothing is listening there yet, is it lost ?
[Friday 24 September 2010] [13:04:58] <praveenbm5> its not lost
[Friday 24 September 2010] [13:05:25] <praveenbm5> queues exist even when the end point doesnt yet exist
[Friday 24 September 2010] [13:05:53] <praveenbm5> i suggest you read the guide on zeromq.org
[Friday 24 September 2010] [13:05:53] <codebeaker> ah, ok - great - i’m actually looking into it for writing a plugin for winamp that fires off a list of whats playing
[Friday 24 September 2010] [13:06:14] <codebeaker> (and I can’t be sure the other app is running)
[Friday 24 September 2010] [13:06:52] <praveenbm5> sorry i got your questiong wrong
[Friday 24 September 2010] [13:07:18] <praveenbm5> you can connect to an end point from a zmq socket before the endpoint exists
[Friday 24 September 2010] [13:07:19] <jhawk28> codebeaker: the new terminology for “upstream” is “push”
[Friday 24 September 2010] [13:07:32] <praveenbm5> but regarding messages
[Friday 24 September 2010] [13:07:38] <codebeaker> no problem, winamp loads .dlls like plugins, and I wanted that plugin to shout out (over zmq) “hey, something played in winamp” - what happens if something isn’t listening ?
[Friday 24 September 2010] [13:08:07] <praveenbm5> push is for parallel worker based execution
[Friday 24 September 2010] [13:08:14] <jhawk28> it sits in the local queue till the socket can connect I believe
[Friday 24 September 2010] [13:08:49] <jhawk28> upstream/downstream == push/pull
[Friday 24 September 2010] [13:09:31] <jhawk28> the only one that doesn’t “keep” the message is the pub socket when noone has connected (I believe)
[Friday 24 September 2010] [13:10:15] <codebeaker> jhawk28: great, and that local queue is transient, in my example - if the closed winamp, and the plugin unloaded - it would be lost, right ?
[Friday 24 September 2010] [13:12:04] <jhawk28> yes, I think so
[Friday 24 September 2010] [13:12:29] <codebeaker> jhawk28: thanks, I’m going to screw around in the Ruby console a little - but that sounds like exactly the behaviour I was looking for
[Friday 24 September 2010] [13:12:41] <jhawk28> you would have to manually persist if you dont want it transient
[Friday 24 September 2010] [13:13:05] <codebeaker> yeah, jhawk28 - I’m looking for it not to persist, or at least to know if it was read off (I suppose I could always examine the local queue)
[Friday 24 September 2010] [13:13:06] <jhawk28> easiest way to see is to download one of the samples and then compile it
[Friday 24 September 2010] [13:13:22] <jhawk28> start them both a couple of times, but in different orders
[Friday 24 September 2010] [13:13:27] <codebeaker> also, lsat question (and I can infer a litlte from the guide) the ipc:// dsn specification, it is portable across windows/unix, I assume ?
[Friday 24 September 2010] [13:13:43] <jhawk28> I havent gotten ipc to work on my windows machine
[Friday 24 September 2010] [13:13:48] <jhawk28> the inproc seems to
[Friday 24 September 2010] [13:14:25] <codebeaker> ahh - I was sortof counting on that - however since it’s trivial, and it’s not really IPC, if it’s two separate apps a .dll in winamp, and something else standalone - maybe IPC is the wrong paradigm now
[Friday 24 September 2010] [13:14:35] <jhawk28> from the man page “The inter-process transport passes messages between local processes using a system-dependent IPC mechanism.”
[Friday 24 September 2010] [13:15:12] <jhawk28> vs “The in-process transport passes messages via memory directly between threads sharing a single MQ context”
[Friday 24 September 2010] [13:15:51] <codebeaker> ah, so IPC is certainly somehow better suited, but a socket would work just as well
[Friday 24 September 2010] [13:18:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [13:19:03] <jhawk28> they are both “sockets”, but the IPC is dependent on the OS
[Friday 24 September 2010] [13:19:08] <codebeaker> ah, ok
[Friday 24 September 2010] [13:19:20] <jhawk28> so if you want cross platform, you probably need inproc
[Friday 24 September 2010] [13:19:30] <guido_g> no
[Friday 24 September 2010] [13:19:37] <codebeaker> inproc wouldn’t work for talking from a winamp process to another process all together
[Friday 24 September 2010] [13:19:44] <guido_g> ipc is between different processes
[Friday 24 September 2010] [13:20:02] <guido_g> on windows, where ipc is not available, use
tcp://127.0.0.1[Friday 24 September 2010] [13:20:03] <codebeaker> I assume I need to go out-of-band if IPC doesn’t work on Windows, and use a real socket, like TCP or sth. that windows supports
[Friday 24 September 2010] [13:20:14] Quit titto has left this server (Quit: Ex-Chat).
[Friday 24 September 2010] [13:20:16] <guido_g> ipc is a real socket
[Friday 24 September 2010] [13:20:26] <codebeaker> ah, but a unix file socket, right ?
[Friday 24 September 2010] [13:20:31] <guido_g> right
[Friday 24 September 2010] [13:20:47] <codebeaker> ok, I assumed (foolishly) that ipc:// on windows would fall-back to tcp://
[Friday 24 September 2010] [13:22:41] <jhawk28> so, inproc is between threads, but not between processes
[Friday 24 September 2010] [13:22:45] <jhawk28> ?
[Friday 24 September 2010] [13:22:52] <codebeaker> jhawk28: certainly sounds like that
[Friday 24 September 2010] [13:24:35] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [13:27:44] <jhawk28> has anyone done the zmq and jzmq build on vs2010 on Win7_64bit?
[Friday 24 September 2010] [13:28:34] <guido_g> have a look at the mailing list, it’s been discussed there afair
[Friday 24 September 2010] [13:38:01] Quit keffo has left this server (Ping timeout: 255 seconds).
[Friday 24 September 2010] [13:41:27] Part mjw9100 has left this channel.
[Friday 24 September 2010] [13:48:02] Quit keffo has left this server (Ping timeout: 264 seconds).
[Friday 24 September 2010] [13:53:11] <jond> codebreaker: ipc isnt implemented on windows, inproc is between threads.
[Friday 24 September 2010] [13:53:20] Part praveenbm5 has left this channel.
[Friday 24 September 2010] [14:01:12] <codebeaker> jond: rgr that (from earlier!) - pretty sure i’ll go with a tcp socket on both platforms
[Friday 24 September 2010] [14:01:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:14:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:23:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:33:26] Quit mytrile has left this server (Quit: Leaving.).
[Friday 24 September 2010] [14:38:47] <jhawk28> hurrah! I got VS2010 to compile zmq and jzmq
[Friday 24 September 2010] [14:38:55] <codebeaker> w00t, well done :-D
[Friday 24 September 2010] [14:48:17] Quit mytrile has left this server (Read error: Operation timed out).
[Friday 24 September 2010] [14:53:02] Quit sustrik has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [15:01:35] Quit xla has left this server (Quit: leaving).
[Friday 24 September 2010] [15:18:40] <rgl> hi
[Friday 24 September 2010] [15:23:09] <codebeaker> hi rgl
[Friday 24 September 2010] [15:33:27] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [15:45:33] Quit starkdg has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [15:47:10] Part mjw9100 has left this channel.
[Friday 24 September 2010] [16:08:53] Quit
mw_ has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [16:18:51] <jond> codebeaker: ipc is faster on linux, than tcp loopback; on solaris it made no difference as solaris loopback is optimised
[Friday 24 September 2010] [16:19:29] <jond> sustrik: how did the presentation go?
[Friday 24 September 2010] [16:55:58] Quit jhawk28 has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [16:56:36] Quit keffo has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [17:06:00] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [17:25:24] Quit codebeaker has left this server (Quit: codebeaker).
[Friday 24 September 2010] [17:26:15] Quit codebeaker has left this server (Client Quit).
[Friday 24 September 2010] [17:27:25] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 24 September 2010] [17:34:07] Quit jhawk28 has left this server (Remote host closed the connection).
[Friday 24 September 2010] [17:40:26] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [17:48:18] Quit ysynopsis has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [18:24:15] Quit rgl has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [18:44:37] Nick _rgl is now known as rgl.
[Friday 24 September 2010] [19:00:43] Quit bennymack has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [19:19:18] Quit mytrile has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [19:20:18] Quit rgl has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [19:20:40] Quit friendshi has left this server (Remote host closed the connection).
[Friday 24 September 2010] [20:47:31] Quit starkdg has left this server (Quit: goodbye).
[Friday 24 September 2010] [21:56:52] Quit iFire has left this server (Read error: Connection reset by peer).
*** on Fri Sep 24 22:16:23 2010
[Friday 24 September 2010] [22:16:23] Topic The channel topic is “Welcome!”.
[Friday 24 September 2010] [22:16:27] Mode Channel modes: no messages from outside, topic protection
[Friday 24 September 2010] [22:16:27] Created This channel was created on 2010-02-05 10:44.
[Friday 24 September 2010] [22:30:23] Quit
mw_ has left this server (Quit:
mw_).
[Friday 24 September 2010] [22:31:40] Quit grahamalot has left this server (Quit: grahamalot).
[Friday 24 September 2010] [23:09:34] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [23:10:42] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [00:05:07] Quit RobertLJ has left this server (Ping timeout: 252 seconds).
[Saturday 25 September 2010] [01:20:48] Quit rgl has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [01:52:35] Quit lluad has left this server (Quit: lluad).
[Saturday 25 September 2010] [02:32:42] <sustrik> Pe_Ell: 2.1 looks more or less stable already. Please, do use it. If any bug is found, let me know.
[Saturday 25 September 2010] [02:36:19] Quit bgranger has left this server (Quit: bgranger).
[Saturday 25 September 2010] [02:50:20] <sustrik> jond: it went well, i’ll post my paper shortly
[Saturday 25 September 2010] [02:50:39] <sustrik> met a lot of linux kernel folks there
[Saturday 25 September 2010] [02:58:35] <Pe_Ell> wow... chapter3 just doubled in size..
[Saturday 25 September 2010] [02:58:51] Quit rgl has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [03:09:18] Quit keffo has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [03:25:07] <Pe_Ell> sustrik, cool. Mostly was interested in accessing the “socket” more directly so there could be some attempt of creting somelike like a Perl AnyEvent wrapper...
[Saturday 25 September 2010] [03:26:37] <Pe_Ell> wow... my english sucks...
[Saturday 25 September 2010] [03:29:47] <sustrik> Pe_Ell: yes, feel free to play with that
[Saturday 25 September 2010] [03:30:09] <sustrik> there’s ZMQ_FD socket option that allows you to do that
[Saturday 25 September 2010] [03:30:33] <Pe_Ell> yeah I was looking over the code and currently reading the guide...
[Saturday 25 September 2010] [03:30:40] <Pe_Ell> pretty entertaining and useful
[Saturday 25 September 2010] [03:30:44] <sustrik> however, it’s a bit more complex than using standard 0mq API
[Saturday 25 September 2010] [03:31:03] <sustrik> you have to keep in mind that:
[Saturday 25 September 2010] [03:31:34] <sustrik> 1. ZMQ_FD can signal even if there’s no message available (on internal 0MQ events such as “connection established”)
[Saturday 25 September 2010] [03:31:47] <sustrik> 2. ZMQ_FD is edge triggered
[Saturday 25 September 2010] [03:32:28] <sustrik> if you don’t care about that, it should work just fine
[Saturday 25 September 2010] [03:32:51] <Pe_Ell> yeah I figured I’d finish the guide before digging into the 2.1 code more...
[Saturday 25 September 2010] [04:09:07] Quit mytrile has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [05:25:29] <pieter_hintjens> Pe_Ell: I moved a lot of Ch3 into Ch2 as well
[Saturday 25 September 2010] [05:41:48] Quit mytrile has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [06:34:27] Quit mytrile has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [07:49:04] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [08:12:25] Quit jsimmons has left this server (Ping timeout: 252 seconds).
[Saturday 25 September 2010] [08:13:18] Quit rossij has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [08:34:48] Quit ysynopsis has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [08:58:48] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [09:33:13] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [09:54:24] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [09:57:04] Quit ysynopsis has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [10:41:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [11:46:14] Quit lluad has left this server (Ping timeout: 264 seconds).
[Saturday 25 September 2010] [11:46:14] Nick
lluad_ is now known as lluad.
*** on Sat Sep 25 13:29:53 2010
[Saturday 25 September 2010] [13:29:53] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [13:29:57] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [13:29:57] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [13:43:50] <Pe_Ell> pieter_hintjens: so I should start over? :) I noticed that it had shifted some, I was going to go look at the earlier chapters to see if I’d missed anything... still got a lot of code to read... :)
[Saturday 25 September 2010] [13:44:26] <pieter_hintjens> Pe_Ell: well, most of the new stuff is at the end of Ch2 and then Ch3
[Saturday 25 September 2010] [13:47:05] <Pe_Ell> yeah I figure I’d like a pretty generic AnyEvent wrapper if it’s reasonable to build... I’m still working out the application design for my project though... I just got distracted with new and shiny.... :D
[Saturday 25 September 2010] [13:57:51] <Pe_Ell> so to make sure I’m not retarded... for pubsub I can make one ZMQ_SUB socket and then just set multiple ZMQ_SUBSCRIPT options on it...
[Saturday 25 September 2010] [13:58:24] <Pe_Ell> err, ZMQ_SUBSCRIBE
[Saturday 25 September 2010] [13:58:45] <Pe_Ell> man I don’t even have a spell checker... not sure why I keep writing out the wrong words...
[Saturday 25 September 2010] [13:59:50] <Pe_Ell> ok man page confirmed what I was thinking... and looks like the string is a prefix so that answered my next question about wildcarding...
[Saturday 25 September 2010] [14:36:32] Quit pieter_hintjens has left this server (Ping timeout: 276 seconds).
[Saturday 25 September 2010] [14:55:25] <lluad> Anyone have any pointers for embedding 0mq into another systems event loop - I’m looking at using it with Qt4.
[Saturday 25 September 2010] [15:13:53] Quit lluad has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [15:13:53] Nick
lluad_ is now known as lluad.
[Saturday 25 September 2010] [15:45:01] <cremes> lluad: take a look at the 2.1 branch specifically ZM_FD and ZM_EVENTS
[Saturday 25 September 2010] [15:45:10] <lluad> Excellent, thanks.
*** on Sat Sep 25 17:20:20 2010
[Saturday 25 September 2010] [17:20:20] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [17:20:24] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [17:20:24] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [17:26:46] Quit
antares_ has left this server (Quit:
antares_).
[Saturday 25 September 2010] [17:45:16] Quit
antares_ has left this server (Quit:
antares_).
[Saturday 25 September 2010] [18:09:36] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Saturday 25 September 2010] [18:27:34] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
*** on Sat Sep 25 21:11:14 2010
[Saturday 25 September 2010] [21:11:14] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [21:11:18] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [21:11:18] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [21:48:28] Quit sbahra has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [21:53:53] Quit pasha_bastov has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [22:33:30] Quit lluad has left this server (Quit: lluad).
[Saturday 25 September 2010] [22:59:26] Quit
pasha_bastov_ has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [23:15:34] Join jdroid- has joined this channel (
~user@74.72.57.182).
[Saturday 25 September 2010] [23:16:25] <jdroid-> hello. i’m reading whitepapers:design-v06. am i right that this is mapped to zeromq v 0.6, making it also quite old?
[Saturday 25 September 2010] [23:27:05] Quit iFire has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [23:37:37] Quit jdroid- has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [23:41:07] Quit bgranger has left this server (Quit: bgranger).
[Saturday 25 September 2010] [23:43:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [01:05:48] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 26 September 2010] [01:49:04] Join sbahra has joined this channel (
~sbahra@pool-74*** on Sun Sep 26 12:46:34 2010
[Sunday 26 September 2010] [12:46:34] Topic The channel topic is “Welcome!”.
[Sunday 26 September 2010] [12:46:38] Mode Channel modes: no messages from outside, topic protection
[Sunday 26 September 2010] [12:46:38] Created This channel was created on 2010-02-05 10:44.
[Sunday 26 September 2010] [13:20:48] Quit bennymack has left this server (Ping timeout: 240 seconds).
[Sunday 26 September 2010] [14:50:48] Quit xla has left this server (Ping timeout: 265 seconds).
[Sunday 26 September 2010] [15:10:08] Quit eintr has left this server (Quit: Leaving.).
[Sunday 26 September 2010] [15:16:52] Quit pasha_bastov has left this server (Quit: Leaving).
[Sunday 26 September 2010] [15:43:34] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * rbe79a9f 10/ (4 files): minor beautifications -
http://bit.ly/cH9kWR[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * rcf815e8 10/ (18 files): new interfaces for timers; the implementation is not changed yet -
http://bit.ly/buVKBu[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r8d7bf66 10/ (13 files): common base for all pollers created; the only thing it handles at the moment is ‘load’ -
http://bit.ly/cqLLHi[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r238640a 10/ (15 files): timers properly implemented -
http://bit.ly/9clGnp[Sunday 26 September 2010] [15:54:10] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Sunday 26 September 2010] [16:19:47] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [17:48:57] Quit GeekGod has left this server (Quit: GeekGod).
[Sunday 26 September 2010] [19:18:20] Quit lotrpy has left this server.
[Sunday 26 September 2010] [19:44:00] <jhawk28> hello, anyone here with windows build experience?
[Sunday 26 September 2010] [19:44:00] Quit zmqircd has left this server (Remote host closed the connection).
[Sunday 26 September 2010] [19:44:35] <jhawk28> I’m having trouble getting the windows code to run on other systems that don’t have VS2008 installed
[Sunday 26 September 2010] [20:05:07] <Zao> jhawk28: Several solutions.
[Sunday 26 September 2010] [20:05:49] <Zao> jhawk28: 1) deploy the runtime via installer. 2) deploy the runtime privately in a subdirectory of your application. 3) build your project against the static runtime, with all the downsides and pitfalls that has.
[Sunday 26 September 2010] [20:06:23] <Zao> Deploying privately with 2008 amounts to grabbing the Microsoft.VC90.CRT directory and putting it next to your executable.
[Sunday 26 September 2010] [20:06:36] <Zao> Deploying via installer can be done either via the standalone installer or via merge modules.
[Sunday 26 September 2010] [20:06:47] <jhawk28> I don’t mind installing the vs2008 redist
[Sunday 26 September 2010] [20:06:51] <Zao> Building against static runtime places severe restrictions to multi-module programs.
[Sunday 26 September 2010] [20:07:11] <Zao> Then install the runtime(s) that your application and your dependencies are built against.
[Sunday 26 September 2010] [20:07:19] <jhawk28> but, I can’t figure out how to have zmq compile against the latest version
[Sunday 26 September 2010] [20:07:19] <Zao> If you’re doing things right, that should be a single runtime version.
[Sunday 26 September 2010] [20:08:47] <jhawk28> btw, this is for the java bindings and the zmq library
[Sunday 26 September 2010] [20:10:19] <jhawk28> but it didn’t change the values in the manifest
[Sunday 26 September 2010] [20:10:46] <Zao> I feel sorry for you. Building against and deploying the right 2008 runtime is great fun.
[Sunday 26 September 2010] [20:10:50] <Zao> Especially with the hotfixes.
[Sunday 26 September 2010] [20:11:57] <jhawk28> :)
[Sunday 26 September 2010] [20:12:01] <jhawk28> :(
[Sunday 26 September 2010] [20:14:32] <jhawk28> new to the whole Visual Studio world so its mostly painful right now
[Sunday 26 September 2010] [20:14:56] <jhawk28> mostly happy that ZMQ compiles out of the box
[Sunday 26 September 2010] [20:16:30] <jhawk28> seems to cover your 3 options
[Sunday 26 September 2010] [20:19:09] Quit xla has left this server (Ping timeout: 265 seconds).
[Sunday 26 September 2010] [20:20:49] <jhawk28> ha, the statically linking only works VS2003 and before based on the comments
[Sunday 26 September 2010] [20:21:29] <jhawk28> and the including the dlls in a sub directory doesn’t work with VS2008-SP1
[Sunday 26 September 2010] [20:23:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [20:43:38] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [20:59:15] Quit bgranger has left this server (Quit: bgranger).
[Sunday 26 September 2010] [21:03:19] Part xrfang has left this channel.
[Sunday 26 September 2010] [22:14:19] Part larrytheliquid has left this channel.
[Sunday 26 September 2010] [22:18:28] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Sunday 26 September 2010] [22:25:28] Quit jhawk28 has left this server (Remote host closed the connection).
[Sunday 26 September 2010] [22:52:02] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [23:12:53] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [23:58:44] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [00:15:22] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [00:21:13] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [00:45:31] Quit mcxx has left this server (Remote host closed the connection).
[Monday 27 September 2010] [01:10:15] Quit pasha_bastov has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [01:11:47] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:21:10] Quit pasha_bastov has left this server (Quit: Leaving).
[Monday 27 September 2010] [01:24:22] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:30:09] Quit guido_g has left this server (Quit: Boom...).
[Monday 27 September 2010] [01:32:49] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:34:01] Quit pasha_bastov has left this server (Ping timeout: 252 seconds).
[Monday 27 September 2010] [01:34:59] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:44:17] Part pieter_hintjens has left this channel.
[Monday 27 September 2010] [01:48:26] Quit zmqircd has left this server (Remote host closed the connection).
[Monday 27 September 2010] [01:52:33] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:53:29] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:53:56] Quit pasha_bastov has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [02:22:49] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [02:39:17] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [03:24:19] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [03:47:37] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [03:56:51] <CIA-20> zeromq2: 03Martin Sustrik 07master * r30c1cba 10/ src/poller_base.cpp : latency optimisation for the case where there are no timers -
http://bit.ly/d8hem9[Monday 27 September 2010] [03:56:51] <CIA-20> zeromq2: 03Martin Sustrik 07master * re2f834d 10/ doc/zmq_getsockopt.txt : ZMQ_FD and ZMQ_EVENTS documentation added -
http://bit.ly/9nWC7q[Monday 27 September 2010] [04:00:08] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [04:32:56] Quit sustrik has left this server (Quit: Leaving).
[Monday 27 September 2010] [04:55:22] Part pasha_bastov has left this channel (“Leaving”).
[Monday 27 September 2010] [04:55:26] Quit plol has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [04:56:52] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [04:57:14] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [05:03:57] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 27 September 2010] [05:17:46] <CIA-20> zeromq2: 03Martin Sustrik 07master * r16c3884 10/ (4 files in 2 dirs): MSVC build fixed -
http://bit.ly/abDASN[Monday 27 September 2010] [05:25:07] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 27 September 2010] [06:05:58] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:19:58] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:26:26] Quit jond has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [06:38:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:40:20] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:42:02] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:47:48] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:50:37] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:52:19] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:55:18] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [07:04:55] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [07:36:09] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [08:11:06] Quit pasha_bastov has left this server (Quit: Leaving).
[Monday 27 September 2010] [08:30:02] Quit plol has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [08:49:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [09:29:25] Quit yawn has left this server (Quit: yawn).
[Monday 27 September 2010] [09:43:47] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [09:44:44] Quit sbahra has left this server (Remote host closed the connection).
[Monday 27 September 2010] [09:47:28] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [09:57:08] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [10:08:45] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [10:28:41] Quit guido_g has left this server (Quit: Boom...).
[Monday 27 September 2010] [10:34:29] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [10:54:21] Quit mytrile has left this server (Quit: Leaving.).
[Monday 27 September 2010] [11:00:08] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [11:12:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:27:38] Quit xla has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [11:28:56] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:29:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 27 September 2010] [11:52:56] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [11:55:02] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:56:03] <shawn42> hello
[Monday 27 September 2010] [11:56:03] Quit zmqircd has left this server (Remote host closed the connection).
[Monday 27 September 2010] [11:57:05] <shawn42> has anyone here done any P2P stuff w/ zeromq?
[Monday 27 September 2010] [12:04:07] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [12:04:07] Nick
drbobbeaty_ is now known as drbobbeaty.
[Monday 27 September 2010] [12:54:26] Quit shawn42 has left this server (Quit: shawn42).
[Monday 27 September 2010] [12:58:31] Quit
antares_ has left this server (Quit:
antares_).
[Monday 27 September 2010] [13:30:28] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [13:57:28] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [14:33:08] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [14:55:58] Quit omarkj has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [15:53:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [16:16:41] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Monday 27 September 2010] [16:24:35] Quit plol has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [16:33:06] Quit
antares_ has left this server (Quit:
antares_).
[Monday 27 September 2010] [16:44:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [16:46:41] Quit ModusPwnens has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [17:02:57] Quit keffo has left this server (Read error: Operation timed out).
[Monday 27 September 2010] [17:05:39] <kristianpaul> Hello
[Monday 27 September 2010] [17:06:48] <kristianpaul> :/
[Monday 27 September 2010] [17:29:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [17:29:39] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 27 September 2010] [17:32:49] Quit zmqircd has left this server (Remote host closed the connection).
*** on Mon Sep 27 17:35:20 2010
[Monday 27 September 2010] [17:35:20] Topic The channel topic is “Welcome!”.
[Monday 27 September 2010] [17:35:24] Mode Channel modes: no messages from outside, topic protection
[Monday 27 September 2010] [17:35:24] Created This channel was created on 2010-02-05 10:44.
[Monday 27 September 2010] [17:37:07] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [17:38:32] Join jond has joined this channel (
~jon@87.74.16.24).
[Monday 27 September 2010] [17:40:13] Quit ModusPwnens has left this server.
[Monday 27 September 2010] [17:50:58] Quit andrewvc has left this server (Ping timeout: 272 seconds).
[Monday 27 September 2010] [18:02:51] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [18:08:21] <ipc> howdy; does anyone know if you can run zmq ‘threadless’; i.e. one process use a TCP-bound zmq socket?
[Monday 27 September 2010] [18:08:53] <ipc> the context docs suggest you shouldn’t init with zero threads unless you’re only using in-process sockets
[Monday 27 September 2010] [18:15:19] Part ipc has left this channel.
[Monday 27 September 2010] [18:33:51] Quit bgranger has left this server (Read error: Operation timed out).
[Monday 27 September 2010] [18:45:19] <kristianpaul> ahh solved
[Monday 27 September 2010] [18:45:24] <kristianpaul> i missed the lib
[Monday 27 September 2010] [18:45:28] <kristianpaul> bye
[Monday 27 September 2010] [18:45:30] Part kristianpaul has left this channel.
[Monday 27 September 2010] [18:57:58] <jhawk28> hello
[Monday 27 September 2010] [19:24:42] Quit aleator has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:24:43] Quit neopallium has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:24:43] Quit Zao has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:53:07] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [20:21:23] Quit andrewvc has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [20:35:53] Quit xla has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [20:51:21] Quit xla has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [20:57:04] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Monday 27 September 2010] [21:16:00] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [21:45:51] Quit jhawk28 has left this server (Remote host closed the connection).
[Monday 27 September 2010] [22:41:48] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [23:39:45] Quit
mw_ has left this server (Quit:
mw_).
[Tuesday 28 September 2010] [00:00:16] Quit grahamalot has left this server (Quit: grahamalot).
[Tuesday 28 September 2010] [01:00:03] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 28 September 2010] [01:15:01] Quit guido_g has left this server (Client Quit).
[Tuesday 28 September 2010] [01:39:30] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rf61921d 10/ src/req.cpp : REQ socket can die when reply is delivered on wrong unerlying connection – fixed -
http://bit.ly/cX2rpW[Tuesday 28 September 2010] [01:46:29] <CIA-20> zeromq2: 03Dhammika Pathirana 07maint * rc1deb22 10/ src/ypipe.hpp : crash when closing an ypipe – fixed -
http://bit.ly/bmxM8U[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Martin Sustrik 07master * rf61921d 10/ src/req.cpp : REQ socket can die when reply is delivered on wrong unerlying connection – fixed -
http://bit.ly/cX2rpW[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Dhammika Pathirana 07master * rc1deb22 10/ src/ypipe.hpp : crash when closing an ypipe – fixed -
http://bit.ly/bmxM8U[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6715f9b 10/ src/ypipe.hpp :
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: Merge branch ‘maint’
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: * maint:
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: crash when closing an ypipe – fixed -
http://bit.ly/cxlL0o[Tuesday 28 September 2010] [01:54:02] Quit plol has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [01:57:13] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [03:18:51] Quit plol has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [03:56:38] Quit neopallium has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [04:03:59] Quit zmqircd has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [04:38:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [05:20:29] <keffo> pieterh, around?
[Tuesday 28 September 2010] [05:21:35] <pieterh> keffo: hi!
[Tuesday 28 September 2010] [05:23:12] <keffo> Busy?
[Tuesday 28 September 2010] [05:24:35] <pieterh> always, but shoot...
[Tuesday 28 September 2010] [05:24:53] <keffo> I’m in the process of solidifying how the network is monitored.. Currently I gather various info & statistics in the loadbalancer which publishes regularly(~5s), and a WPF app subscribing and displaying nice graphs etc
[Tuesday 28 September 2010] [05:25:19] <pieterh> sounds good
[Tuesday 28 September 2010] [05:25:21] <keffo> but I cant really figure out a good way of limiting it.. I dont want to send a complete state every 5s
[Tuesday 28 September 2010] [05:25:22] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 28 September 2010] [05:25:30] <pieterh> how large is the state?
[Tuesday 28 September 2010] [05:26:24] <keffo> Depends, both on what info I decide to publish.. I’d like load, bandwidth usage, connected nodes and their respective stats(cpu/ram/etc), but also an overview of what is happenening, as well as more detailed info for each worker
[Tuesday 28 September 2010] [05:27:07] <pieterh> estimated size? in bytes?
[Tuesday 28 September 2010] [05:27:21] <keffo> geesh, no idea, not in the mb range at least :)
[Tuesday 28 September 2010] [05:27:40] <pieterh> if you have no idea, it’s not sensible to think about optimizing it
[Tuesday 28 September 2010] [05:27:59] <pieterh> so do a back-of-envelope calculation and come up with a figure...
[Tuesday 28 September 2010] [05:28:11] <keffo> I wondered if it was sound to have the monitoring app poll a complete current-state at startup, and then depend on a persistent sub-forwarder to handle deltastates? But that sounds very complicated
[Tuesday 28 September 2010] [05:28:48] <keffo> I can’t really guestimate, the number of nodes can range from the local setup I have here of a few machines, to much larger..
[Tuesday 28 September 2010] [05:28:57] <pieterh> ...
[Tuesday 28 September 2010] [05:29:05] <pieterh> how big is “much larger”?
[Tuesday 28 September 2010] [05:29:18] <keffo> ideally wan :)
[Tuesday 28 September 2010] [05:29:30] <pieterh> please stick a number onto it...
[Tuesday 28 September 2010] [05:29:38] <keffo> but being realistic, perhaps around 20?
[Tuesday 28 September 2010] [05:29:51] <pieterh> and how large would the state be per worker?
[Tuesday 28 September 2010] [05:30:14] <pieterh> please stick a number onto it...
[Tuesday 28 September 2010] [05:30:29] <pieterh> then multiply the two numbers and add something for the overview
[Tuesday 28 September 2010] [05:30:31] <keffo> Basic info(linpack measurements), uptime, average load, around that
[Tuesday 28 September 2010] [05:30:51] <pieterh> come back when you have a total in KB per 5 seconds, a’ight?
[Tuesday 28 September 2010] [05:32:02] <keffo> That’s not what I’m interested in, the assumption here is that no data is published that isn’t “needed”, but I would like to figure out the most efficient means of passing around that data
[Tuesday 28 September 2010] [05:32:29] <keffo> basically how to monitor a distributed system with the least amount of overhead as possible..
[Tuesday 28 September 2010] [05:32:45] <pieterh> this is for research purposes rather than an actual use case...
[Tuesday 28 September 2010] [05:32:53] <keffo> (regardless of what the data actually is)
[Tuesday 28 September 2010] [05:33:03] <keffo> well both I guess, research first, use later :)
[Tuesday 28 September 2010] [05:33:23] <pieterh> well, you can wait until Ch4 of the Guide if you want to
[Tuesday 28 September 2010] [05:33:25] <keffo> Surely this problem has been dealt with before, as loadbalancing has :)
[Tuesday 28 September 2010] [05:33:28] <pieterh> but here is how I’d do it...
[Tuesday 28 September 2010] [05:33:37] <pieterh> - maintain state in the publisher
[Tuesday 28 September 2010] [05:33:47] <pieterh> - apply updates to state and publish updates to pub socket
[Tuesday 28 September 2010] [05:34:01] <pieterh> - in subscriber, request state via req/rep socket
[Tuesday 28 September 2010] [05:34:07] <pieterh> - and also subscribe to updates
[Tuesday 28 September 2010] [05:34:10] <pieterh> - queue incoming updates
[Tuesday 28 September 2010] [05:34:22] <pieterh> - as soon as state arrives, apply updates to state and continue to do this
[Tuesday 28 September 2010] [05:34:22] <keffo> m, that’s what I was thinking as well.
[Tuesday 28 September 2010] [05:35:17] <pieterh> i think it’s robust but need to prove it
[Tuesday 28 September 2010] [05:35:17] <keffo> Things do start to get hairy if the monitor app starts to depend on delta-updates though, like “join/part” of nodes
[Tuesday 28 September 2010] [05:35:56] <keffo> you’re doing this type of stuff for ch 4?
[Tuesday 28 September 2010] [05:35:59] <pieterh> yup
[Tuesday 28 September 2010] [05:36:05] <pieterh> stateful pubsub
[Tuesday 28 September 2010] [05:36:42] <pieterh> or whatever this is properly called...
[Tuesday 28 September 2010] [05:36:55] <keffo> I’ll try it out, see if it behaves well.. :)
[Tuesday 28 September 2010] [05:37:12] <pieterh> feel free to write it up as a recipe or code sample
[Tuesday 28 September 2010] [05:37:18] <pieterh> if i can reuse that for the guide it’ll save me time
[Tuesday 28 September 2010] [05:37:37] <keffo> I’ll let you know for sure yeah
[Tuesday 28 September 2010] [05:38:06] <pieterh> the main reason for this is not so much to save network bandwidth but to allow realtime updates
[Tuesday 28 September 2010] [05:38:21] <keffo> I was aso thinking about ‘history-nodes’ as well.. something appealing about that, someone who keeps track of what’s going on, but isn’t directly part -of- the system
[Tuesday 28 September 2010] [05:38:48] <pieterh> indeed, this work could be totally outsourced to a stateful device
[Tuesday 28 September 2010] [05:38:50] <keffo> non-gonzo network monitoring
[Tuesday 28 September 2010] [05:39:08] <pieterh> with some notion of state + patches
[Tuesday 28 September 2010] [05:39:47] <pieterh> like pair/value updates
[Tuesday 28 September 2010] [05:39:52] <pieterh> hmm, nice
[Tuesday 28 September 2010] [05:39:59] <pieterh> it’s a distributed cache
[Tuesday 28 September 2010] [05:40:08] Quit mytrile has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [05:40:14] <keffo> yes, distributed tuple-storage
[Tuesday 28 September 2010] [05:40:19] <pieterh> yup, that’s the thing
[Tuesday 28 September 2010] [05:40:54] <keffo> I cant help but thinking most source control software face much of the same issues here
[Tuesday 28 September 2010] [05:40:59] <pieterh> in the general solution any node can update its cache
[Tuesday 28 September 2010] [05:41:21] <pieterh> it is a very useful general solution to state distribution
[Tuesday 28 September 2010] [05:41:27] <keffo> oh yeah, one can think of it as cache hits and misses, makes things clearer
[Tuesday 28 September 2010] [05:42:39] <pieterh> i’d start with a simple model, one publisher, many subscribers, pair/value updates
[Tuesday 28 September 2010] [05:43:05] <keffo> It reuses the same mechanisms as the rest
[Tuesday 28 September 2010] [05:44:59] <keffo> maybe I could contribute some C# stuff to go along with ch 4.. I do find it quite lacking on the site..
[Tuesday 28 September 2010] [05:45:56] <pieterh> keffo: we’ve gotten one C# example yesterday, but others would be great
[Tuesday 28 September 2010] [05:46:13] <pieterh> do start at ch1 if you could, it’s quite trivial stuff but useful to newcomers
[Tuesday 28 September 2010] [05:46:34] <keffo> yeah..
[Tuesday 28 September 2010] [05:46:50] <keffo> need a complete binding, for starters :)
[Tuesday 28 September 2010] [05:47:00] <pieterh> the binding is not complete?
[Tuesday 28 September 2010] [05:47:28] <keffo> no, there was no poll for example?
[Tuesday 28 September 2010] [05:47:34] <keffo> or did I somehow get an old version?
[Tuesday 28 September 2010] [05:48:28] <keffo> nope..
[Tuesday 28 September 2010] [05:48:53] <pieterh> who maintains this binding?
[Tuesday 28 September 2010] [05:49:00] <pieterh> ask them to fix it or submit a patch
[Tuesday 28 September 2010] [05:49:07] <keffo> not sure, says sustrik last commit I guess
[Tuesday 28 September 2010] [05:49:32] <pieterh> hmm, the owner of every project should be WRITTEN IN HUGE LETTERS
[Tuesday 28 September 2010] [05:49:41] <pieterh> otherwise it’s kind of dead by definition
[Tuesday 28 September 2010] [05:49:51] <pieterh> nzmq is something layered on top afaics
[Tuesday 28 September 2010] [05:49:58] <pieterh> different API
[Tuesday 28 September 2010] [05:50:05] <keffo> yes, but it does the same lowlevel binding of the dll
[Tuesday 28 September 2010] [05:50:44] <pieterh> a’ight
[Tuesday 28 September 2010] [05:51:10] <keffo> So merging them shouldnt be very difficult
[Tuesday 28 September 2010] [05:51:11] * pieterh is wondering who pays trolls to come talk rubbish whenever there’s a license thread
[Tuesday 28 September 2010] [05:51:22] <keffo> on the list?
[Tuesday 28 September 2010] [05:51:40] <pieterh> “What’s the latest version of ZeroMQ that’s not LGPL? I need to static link in commercial projects and the LGPL is not an option. “
[Tuesday 28 September 2010] [05:53:18] <pieterh> not a single sensible comment on that thread, just trolls coming to complain that [sic] switching to LGPL will kill 0MQ...
[Tuesday 28 September 2010] [05:53:58] <keffo> will it? =)
[Tuesday 28 September 2010] [05:54:18] <pieterh> oh, yes, of course...
[Tuesday 28 September 2010] [05:54:38] <pieterh> that’s why we have to go back in time and switch to the Microsoft Open Software License or whatever...
[Tuesday 28 September 2010] [05:55:22] <keffo> licensing is tricky business, it went and got itself hugely complicated
[Tuesday 28 September 2010] [05:55:31] <keffo> I prefer “mine” and “public domain” :)
[Tuesday 28 September 2010] [05:55:41] <pieterh> it’s not really tricky, just politically sensitive because it involves so much money
[Tuesday 28 September 2010] [05:56:13] <pieterh> every license is the contract on which the community grows
[Tuesday 28 September 2010] [05:56:21] <pieterh> LGPL and GPL are IME proven beyond a reasonable doubt to be the most effective contracts
[Tuesday 28 September 2010] [05:56:27] <pieterh> because they make it impossible to cheat
[Tuesday 28 September 2010] [05:56:34] <pieterh> end.
[Tuesday 28 September 2010] [05:56:51] <PerfDave> Not impossible, see gpl-violations.org. But very difficult ;)
[Tuesday 28 September 2010] [05:57:05] <pieterh> impossible in any sustainable sense
[Tuesday 28 September 2010] [05:57:15] <pieterh> and GPLv3 closed the loopholes people found in GPLv2
[Tuesday 28 September 2010] [05:57:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 28 September 2010] [05:57:15] <keffo> who has the time anyway :)
[Tuesday 28 September 2010] [05:57:23] <pieterh> oh, people love to cheat
[Tuesday 28 September 2010] [05:57:41] <pieterh> but communities die when they get parasited
[Tuesday 28 September 2010] [05:58:09] <pieterh> so these trolls come and complain that LGPL will kill the community when in fact it creates it
[Tuesday 28 September 2010] [05:58:14] <pieterh> sigh.
[Tuesday 28 September 2010] [05:58:20] <keffo> :)
[Tuesday 28 September 2010] [06:03:22] <keffo> what’s the most elaborate zmq based project anyway?
[Tuesday 28 September 2010] [06:07:16] <pieterh> keffo: wow, there are some very elaborate ones out there
[Tuesday 28 September 2010] [06:07:32] <pieterh> but most are so secret that I’d have to kill you after explaining them
[Tuesday 28 September 2010] [06:07:58] <keffo> Oh I just mean scale, not impl. details :)
[Tuesday 28 September 2010] [06:08:12] <keffo> I’d like to know what level I’m at :)
[Tuesday 28 September 2010] [06:08:46] <pieterh> scale: hundreds to thousands of nodes
[Tuesday 28 September 2010] [06:09:03] <pieterh> multiple data centers
[Tuesday 28 September 2010] [06:10:00] <keffo> are they mostly about shuffling data around, or generic compute clusters?
[Tuesday 28 September 2010] [06:10:17] <pieterh> both cases
[Tuesday 28 September 2010] [06:10:47] <keffo> interesting stuff
[Tuesday 28 September 2010] [06:10:56] <pieterh> it’s also growing rapidly
[Tuesday 28 September 2010] [06:11:05] <keffo> I would assume so!
[Tuesday 28 September 2010] [06:11:16] <pieterh> the first 0MQ projects a year or two ago were maybe 10 nodes
[Tuesday 28 September 2010] [06:11:36] <pieterh> i’d say the scale is growing x10 every six months or so
[Tuesday 28 September 2010] [06:12:20] <keffo> It would be nice if the license included algorithmic contributions as opposed to solely sourcecode :)
[Tuesday 28 September 2010] [06:12:46] <pieterh> well, ideas cannot be copyrighted
[Tuesday 28 September 2010] [06:13:04] <keffo> A lot of information was probably gathered during development of those, which ideally should be shared :)
[Tuesday 28 September 2010] [06:13:12] <pieterh> well
[Tuesday 28 September 2010] [06:13:30] <pieterh> whenever possible we do move experience into the open source layers
[Tuesday 28 September 2010] [06:14:07] <pieterh> however there are often valuable business secrets in these algorithms
[Tuesday 28 September 2010] [06:14:23] <pieterh> obviously we do not consider sharing those
[Tuesday 28 September 2010] [06:14:47] <keffo> I was more thinking along the lines of “this common-practice method breaks down under these conditions” etc
[Tuesday 28 September 2010] [06:15:00] <pieterh> this is what the user guide will eventually cover
[Tuesday 28 September 2010] [06:15:36] <pieterh> at least the more common cases
[Tuesday 28 September 2010] [06:15:52] <pieterh> we can also try to document some of the higher level patterns as protocols
[Tuesday 28 September 2010] [06:15:53] <keffo> Some of the things I’ve found annoyingly void so far has been the loadbalancing(which is now covered well enough), and also recursive behaviour, which I think I’ve solved
[Tuesday 28 September 2010] [06:16:10] <pieterh> have you been using the custom routing from Ch3?
[Tuesday 28 September 2010] [06:16:30] <keffo> Yeah, pretty much, but with prioqueues
[Tuesday 28 September 2010] [06:16:37] <pieterh> what is a prioqueue?
[Tuesday 28 September 2010] [06:16:44] <keffo> both for incoming tasks and outgoing results
[Tuesday 28 September 2010] [06:16:50] <keffo> priority queue
[Tuesday 28 September 2010] [06:16:57] <pieterh> ah, so queues in your broker rather than using the socket queues
[Tuesday 28 September 2010] [06:17:03] <keffo> indeed
[Tuesday 28 September 2010] [06:17:17] <pieterh> i just didn’t want to start mucking with data structures
[Tuesday 28 September 2010] [06:17:23] <pieterh> but I think it’s inevitable
[Tuesday 28 September 2010] [06:17:41] <pieterh> i already had to define a zmsg class, will probably define a zqueue class as well
[Tuesday 28 September 2010] [06:17:51] <keffo> messages and tasks have priorities, as well as workers based on a mix of scimark(hardware) and something like a running average network ‘behaviour’
[Tuesday 28 September 2010] [06:17:57] <pieterh> right
[Tuesday 28 September 2010] [06:18:40] <keffo> Another thing I’ve been struggling to figure out a ‘pretty’ solution to is how to make workers present their updates when doing long running jobs..
[Tuesday 28 September 2010] [06:18:58] <pieterh> intermediate updates?
[Tuesday 28 September 2010] [06:18:59] <keffo> That’ sortof ties into what we talked about earlier with the Monitoring app
[Tuesday 28 September 2010] [06:19:00] <keffo> yeah
[Tuesday 28 September 2010] [06:19:06] <pieterh> two threads in the workers, I assume
[Tuesday 28 September 2010] [06:19:17] Quit neopallium has left this server (Read error: Operation timed out).
[Tuesday 28 September 2010] [06:19:17] <pieterh> workers as micro clusters
[Tuesday 28 September 2010] [06:19:31] <pieterh> it’s fractal :-)
[Tuesday 28 September 2010] [06:19:37] <pieterh> every node can be a cluster of nodes
[Tuesday 28 September 2010] [06:19:40] <keffo> leaving it solely up to the designer of the job (ie, they post progress at will) leads to abuse most likely
[Tuesday 28 September 2010] [06:20:07] <pieterh> sounds like you’re solving a lot of interesting problems
[Tuesday 28 September 2010] [06:20:11] <keffo> indeed :)
[Tuesday 28 September 2010] [06:20:21] <keffo> most interesting of all is the fact that a job can post new jobs :)
[Tuesday 28 September 2010] [06:20:22] <pieterh> you should write about it, if you can
[Tuesday 28 September 2010] [06:20:27] <keffo> that’s a brain teaser if anything :)
[Tuesday 28 September 2010] [06:20:28] <keffo> yeah
[Tuesday 28 September 2010] [06:20:38] <keffo> It’s becoming quite large to be honest :)
[Tuesday 28 September 2010] [06:20:46] <pieterh> well, stack-based simulated recursion is an old technique
[Tuesday 28 September 2010] [06:20:50] <keffo> but it’s working well
[Tuesday 28 September 2010] [06:20:56] <pieterh> it’s how we used to do quicksort in cobol
[Tuesday 28 September 2010] [06:21:04] <keffo> stack-based?
[Tuesday 28 September 2010] [06:21:17] <pieterh> your prioqueue can also be a stack
[Tuesday 28 September 2010] [06:21:22] <pieterh> you can push jobs to the front
[Tuesday 28 September 2010] [06:21:28] <pieterh> or to the back
[Tuesday 28 September 2010] [06:21:32] <keffo> ah yeah, that’s the priority of the messages :)
[Tuesday 28 September 2010] [06:21:41] <pieterh> that’s how you simulate recursion
[Tuesday 28 September 2010] [06:21:43] <keffo> he deeper the recursion, the higher the priority
[Tuesday 28 September 2010] [06:21:58] <pieterh> priority is perhaps the wrong metaphor
[Tuesday 28 September 2010] [06:22:12] <pieterh> in fact it’s “push these child jobs” followed by “pop next job and execute”
[Tuesday 28 September 2010] [06:24:09] <pieterh> If you look at section 3, you see how Quicksort (recursion) works using a stack
[Tuesday 28 September 2010] [06:24:33] <keffo> My solution was to have each worker-node (which owns one worker-process per cpu-core) can request to have an additional worker spawned while it sleeps.. So when it posts a child-job, it does so with a higher priority
[Tuesday 28 September 2010] [06:24:50] <keffo> interesting
[Tuesday 28 September 2010] [06:25:25] <pieterh> Leif and I developed these techniques in the 80’s...
[Tuesday 28 September 2010] [06:25:35] <keffo> how old are you? =)
[Tuesday 28 September 2010] [06:25:38] <pieterh> it’s very basic but I think it maps correctly to recursive messaging
[Tuesday 28 September 2010] [06:25:47] <pieterh> not so old
[Tuesday 28 September 2010] [06:25:51] <pieterh> :-)
[Tuesday 28 September 2010] [06:25:58] <pieterh> 47, to be accurate
[Tuesday 28 September 2010] [06:26:17] <keffo> that was grad student times then? =)
[Tuesday 28 September 2010] [06:26:34] <pieterh> nope, first job developing tools for large software houses
[Tuesday 28 September 2010] [06:26:45] <pieterh> event-driven concurrency in cobol
[Tuesday 28 September 2010] [06:27:35] <keffo> obol of all things :)
[Tuesday 28 September 2010] [06:27:57] <keffo> there’s a scary amount of need for cobol developers now
[Tuesday 28 September 2010] [06:28:36] <pieterh> lol...
[Tuesday 28 September 2010] [06:28:54] <pieterh> we used to train people to become cobol developers in like 3 weeks
[Tuesday 28 September 2010] [06:29:13] <keffo> That’s even more scary :)
[Tuesday 28 September 2010] [06:29:22] <keffo> like todays php crowd I guess?
[Tuesday 28 September 2010] [06:29:29] <pieterh> hmm, yeah, I guess
[Tuesday 28 September 2010] [06:29:53] <pieterh> Cobol was good for mediocre programmers, they could make stuff that worked, and didn’t kill the system
[Tuesday 28 September 2010] [06:30:13] <keffo> I’d be very afraid if my bank announced they moved from cobol to php :)
[Tuesday 28 September 2010] [06:30:26] <pieterh> indeed
[Tuesday 28 September 2010] [06:30:48] <keffo> or voting/pacemakers :)
[Tuesday 28 September 2010] [06:44:44] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 28 September 2010] [06:57:18] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [07:01:47] Quit keffo has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [07:28:03] Quit bbigras has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [07:33:31] <CIA-20> jzmq: 03Stefan Majer 07master * rd46166f 10/ src/org/zeromq/ZMQ.java : Merged from upstream -
http://bit.ly/c9pkwW[Tuesday 28 September 2010] [07:33:31] <CIA-20> jzmq: 03Stefan Majer 07master * r05b384d 10/ src/org/zeromq/ZMQ.java : Reduced duplicate Javadoc comments by references to the corresponding setter. -
http://bit.ly/dvaPdr[Tuesday 28 September 2010] [07:33:39] <CIA-20> jzmq: 03Stefan Majer 07master * r1a98406 10/ src/org/zeromq/ZMQ.java : References to the man pages to further clarify the Javadoc. -
http://bit.ly/c5FVwB[Tuesday 28 September 2010] [08:13:38] Join plq has joined this channel (
~plq@78.189.30.218).
[Tuesday 28 September 2010] [08:33:56] Join eintr has joined this channel (
~dln@88.80.182.68).
[Tuesday 28 September 2010] [08:40:18] Quit plq has left this server (Quit: Leaving).
[Tuesday 28 September 2010] [08:49:48] Quit mytrile has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [08:55:33] Quit plol has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [09:13:59] Part plol has left this channel.
[Tuesday 28 September 2010] [09:27:54] <CIA-20> zeromq2: 03Gonzalo Diethelm 07master * r87beaaa 10/ (14 files in 4 dirs): ZMQ_TYPE socket option added -
http://bit.ly/b3HZYe[Tuesday 28 September 2010] [09:30:58] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [09:32:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * r10bb9d0 10/ AUTHORS : Dhammika Pathirana was missing from the AUTOHRS file for some reason – fixed -
http://bit.ly/cz4TZ7[Tuesday 28 September 2010] [09:51:49] Quit sbahra has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [10:07:59] Quit xla has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [10:15:38] Quit jond has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [10:39:50] <CIA-20> zeromq2: 03Steven McCoy 07master * r00cd7d4 10/ (7 files in 3 dirs): Upgrade to OpenPGM-5.0.78 -
http://bit.ly/aKqJjZ[Tuesday 28 September 2010] [10:42:13] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: 03Steven McCopy 07master * r1dc4531 10/ (4 files): (log message trimmed)
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Add assertions to check for OpenPGM calls with invalid parameters.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Assertion to check that pgm_getaddrinfo is actually returning something.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Missing pgm_connect call.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Typo on TOS causing immediate abort.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Placeholder calls for timeouts whilst continuing spin loop functionality.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * OpenPGM v5 now supports reference counting so remove init checks.
[Tuesday 28 September 2010] [11:05:07] Quit zmqircd has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [11:26:22] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 28 September 2010] [11:28:03] <psino> I’m having some issues with understanding how a PUSH socket with no downstream nodes is supposed to behave. There seem to be a difference between the sockets whether they have been used as .bind or .connect
[Tuesday 28 September 2010] [11:28:53] <psino> the connect_socket sends the data (send does not block) even if there are no downstream nodes, but the bind_socket blocks
[Tuesday 28 September 2010] [11:30:48] <psino> is this the expected behaviour? from what I understand after reading
http://api.zeromq.org/zmq_socket.html is that both should have been blocking
[Tuesday 28 September 2010] [11:31:07] <psino> As its said here: “When a ZMQ_PUSH socket enters an exceptional state due to having reached the high water mark for all downstream nodes, or if there are no downstream nodes at all, then any zmq_send(3) operations on the socket shall block until the exceptional state ends or at least one downstream node becomes available for sending; messages are not discarded.”
[Tuesday 28 September 2010] [11:37:27] <sustrik> psino: yes. it works that way
[Tuesday 28 September 2010] [11:38:02] <sustrik> on connect, socket can immediately create a queue to store messages in – even before actual connection is established
[Tuesday 28 September 2010] [11:38:41] <psino> so the manual is incorrect/imprecise?
[Tuesday 28 September 2010] [11:39:10] <sustrik> when binding, the socket has to wait till peers connect, it cannot create a queue itself (it doesn’t even know whether there’ll be a connection in the future)
[Tuesday 28 September 2010] [11:39:43] <sustrik> imprecise, i would say
[Tuesday 28 September 2010] [11:39:55] <sustrik> connect assumed that there’s a peer
[Tuesday 28 September 2010] [11:40:06] <sustrik> exactly 1 peer
[Tuesday 28 September 2010] [11:40:11] <sustrik> bind makes no such assumption
[Tuesday 28 September 2010] [11:41:28] <psino> hmm
[Tuesday 28 September 2010] [11:52:05] <jashmenn> hey can anyone give me some guidance on socket migration between threads
[Tuesday 28 September 2010] [11:52:15] <jashmenn> i’m having a tough time figuring out how that works
[Tuesday 28 September 2010] [11:52:30] <jashmenn> basically i want to close a socket from a thread different than the thread that started it
[Tuesday 28 September 2010] [11:58:14] <mato> sustrik: yo
[Tuesday 28 September 2010] [11:58:48] Quit eintr has left this server (Quit: Leaving.).
[Tuesday 28 September 2010] [11:59:45] <sustrik> mato: hi
[Tuesday 28 September 2010] [11:59:57] <mato> sustrik: i would have appreciated a heads up on the OpenPGM commits
[Tuesday 28 September 2010] [12:00:09] <mato> sustrik: it does touch the build system, of which you declared me the maintainer :-)
[Tuesday 28 September 2010] [12:00:16] <sustrik> ah
[Tuesday 28 September 2010] [12:00:22] <sustrik> i blindly applied the patches :|
[Tuesday 28 September 2010] [12:00:29] <mato> also, isn’t the author of those commits steve mccoy?
[Tuesday 28 September 2010] [12:00:36] <sustrik> yes, he is
[Tuesday 28 September 2010] [12:00:47] <mato> ah, it’s there
[Tuesday 28 September 2010] [12:00:56] <mato> ‘cept you have a typo
[Tuesday 28 September 2010] [12:00:59] <mato> but that’s my fault
[Tuesday 28 September 2010] [12:01:01] <sustrik> ?
[Tuesday 28 September 2010] [12:01:07] * mato is documenting the process
[Tuesday 28 September 2010] [12:01:20] <mato> sustrik: you should not be typing author names manually if at all possible
[Tuesday 28 September 2010] [12:01:32] <sustrik> no idea how to do it automatically
[Tuesday 28 September 2010] [12:01:38] <mato> git-am :-)
[Tuesday 28 September 2010] [12:02:08] <sustrik> good god!
[Tuesday 28 September 2010] [12:02:26] <mato> sustrik: anyway, please do ask for review before committing stuff that touches other people’s designated areas of responsibility
[Tuesday 28 September 2010] [12:02:37] <sustrik> anyway, can you check the part that touches the build system post hoc?
[Tuesday 28 September 2010] [12:02:42] <mato> yes, i can, i will
[Tuesday 28 September 2010] [12:02:47] <sustrik> thanks
[Tuesday 28 September 2010] [12:02:51] <mato> but please don’t do that in future :-)
[Tuesday 28 September 2010] [12:02:54] <mato> including e.g. docs
[Tuesday 28 September 2010] [12:02:56] <sustrik> sure
[Tuesday 28 September 2010] [12:03:03] <sustrik> eek
[Tuesday 28 September 2010] [12:03:07] <mato> eek?
[Tuesday 28 September 2010] [12:03:12] <sustrik> i’ve just committed gonzalos patch
[Tuesday 28 September 2010] [12:03:22] <sustrik> which touchs docs
[Tuesday 28 September 2010] [12:03:36] <sustrik> ZMQ_TYPE socket opt description
[Tuesday 28 September 2010] [12:03:42] <mato> it also has a commit to .gitignore for bin/ for some reason
[Tuesday 28 September 2010] [12:03:51] <mato> as part of gonzalo’s patch
[Tuesday 28 September 2010] [12:03:55] <sustrik> hm
[Tuesday 28 September 2010] [12:04:12] <sustrik> it shouldn’t be there but once it’s there it doesn’t hurt
[Tuesday 28 September 2010] [12:04:15] <mato> sure
[Tuesday 28 September 2010] [12:04:24] <mato> but my point is be careful about what is in a patch
[Tuesday 28 September 2010] [12:04:43] <sustrik> what are we going to do with patches that intersect different areas of functionality?
[Tuesday 28 September 2010] [12:04:53] <sustrik> there’s no clear committer there
[Tuesday 28 September 2010] [12:05:00] <mato> ask the interested parties for review
[Tuesday 28 September 2010] [12:05:03] <mato> that’s how it normally works
[Tuesday 28 September 2010] [12:05:10] <sustrik> ok, will do
[Tuesday 28 September 2010] [12:05:35] <mato> also, if you want my attention quickly a good way to do that is to Cc: me directly as well as sending email to the list
[Tuesday 28 September 2010] [12:05:43] <sustrik> ack
[Tuesday 28 September 2010] [12:05:51] <mato> that way it lands in my INBOX rather than in the auto-filtered-away list folder :-)
[Tuesday 28 September 2010] [12:35:18] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 28 September 2010] [12:42:49] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * re3e6b7f 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java): Added support for getType() and added some missing constants. -
http://bit.ly/at8wUF[Tuesday 28 September 2010] [12:43:04] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [12:57:49] Quit psino has left this server (Quit: psino).
[Tuesday 28 September 2010] [13:02:16] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [13:03:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [13:12:31] Quit psino has left this server (Client Quit).
[Tuesday 28 September 2010] [13:25:37] Quit keffo has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [13:30:21] <vharron> Is there a list of projects using zeromq? I’m trying to understand what it does well by looking at the problems it solves.
[Tuesday 28 September 2010] [13:52:22] <drbobbeaty> vharron: I can’t give you a list of projects, but I can tell you about mine. I’m using ZMQ as a transport layer using OpenPGM and the reliable multicast. I’m using this because what I need is a transport more than a full-fledged messaging system like Tibco or 29West. It’s working out wonderfully in this capacity and we’re moving a lot of messages through the system.
[Tuesday 28 September 2010] [13:53:36] <drbobbeaty> It’s not a real full-blown messaging system - but you can build one with it. I just see it’s real target as a very advanced transport system with a lot of flexibility.
[Tuesday 28 September 2010] [14:04:24] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 28 September 2010] [14:34:33] Quit vharron has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [15:16:22] Quit gavinstark has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [15:51:58] Join jond has joined this channel (
~jon@87.74.16.24).
[Tuesday 28 September 2010] [15:55:18] Quit
mw_ has left this server (Quit:
mw_).
[Tuesday 28 September 2010] [16:20:41] Quit zirpu has left this server (Quit: leaving).
[Tuesday 28 September 2010] [16:23:14] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Tuesday 28 September 2010] [16:29:44] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [16:47:54] <CIA-20> zeromq2: 03Steven McCoy 07master * ra729357 10/ (6 files): more fixes to (e)pgm transport -
http://bit.ly/9E1o2T[Tuesday 28 September 2010] [16:50:08] Quit zmqircd has left this server (Ping timeout: 245 seconds).
*** on Tue Sep 28 16:57:48 2010
[Tuesday 28 September 2010] [16:57:48] Topic The channel topic is “Welcome!”.
[Tuesday 28 September 2010] [16:57:52] Mode Channel modes: no messages from outside, topic protection
[Tuesday 28 September 2010] [16:57:52] Created This channel was created on 2010-02-05 10:44.
[Tuesday 28 September 2010] [17:02:36] Quit zmqircd has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [17:29:16] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 28 September 2010] [17:54:10] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [18:15:07] Quit jashmenn has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [18:28:15] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Tuesday 28 September 2010] [19:14:03] <lluad> If I have multiple clients connecting to a single server over TCP, is there a good idiom for the server to send an unsolicited message to just one of the clients?
[Tuesday 28 September 2010] [19:18:21] Quit rgl has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [19:20:50] <Zao> lluad: When connecting a client, send a connection string to the server if it needs to communicate with you?
[Tuesday 28 September 2010] [19:24:16] Quit lluad has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [19:24:16] Nick
lluad_ is now known as lluad.
[Tuesday 28 September 2010] [19:38:31] Quit lluad has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [19:38:31] Nick
lluad_ is now known as lluad.
[Tuesday 28 September 2010] [20:29:07] Quit rbancroft has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [20:34:15] <novas0x2a> is there an example for how to use zmq_poll properly? This code (where i think i’m using it right?) crashes reliably:
http://codepad.org/uyAlBSOw[Tuesday 28 September 2010] [20:35:28] <novas0x2a> oh wow, i’m lame... ignore that code for a sec..
[Tuesday 28 September 2010] [20:37:54] Quit novas0x2a has left this server (Quit: Leaving.).
[Tuesday 28 September 2010] [21:11:27] Quit iFire has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [21:12:06] Quit miguelito has left this server (Read error: Connection reset by peer).
[Tuesday 28 September 2010] [21:17:59] Quit xla has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [21:24:18] Quit iFire` has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [21:33:26] Quit
RobertLJ_ has left this server (Read error: Operation timed out).
[Tuesday 28 September 2010] [22:28:08] Part neopallium has left this channel (“Leaving”).
[Tuesday 28 September 2010] [22:45:45] Quit lluad has left this server (Quit: lluad).
[Tuesday 28 September 2010] [23:11:38] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Tuesday 28 September 2010] [23:13:49] Quit lluad has left this server (Quit: lluad).
[Wednesday 29 September 2010] [01:11:06] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [01:14:37] Quit iFire has left this server (Remote host closed the connection).
[Wednesday 29 September 2010] [01:32:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 29 September 2010] [01:54:54] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 29 September 2010] [02:11:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 29 September 2010] [02:16:02] Quit rgl has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [02:19:04] Quit jond has left this server (Quit: Leaving).
[Wednesday 29 September 2010] [02:19:38] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [04:03:29] Join eintr has joined this channel (
~dln@88.80.182.68).
[Wednesday 29 September 2010] [04:09:54] <CIA-20> zeromq2: 03Ilja Golshtein 07master * r4482dd7 10/ (AUTHORS include/zmq.hpp): C++ binding destructors don’t throw exceptions -
http://bit.ly/9r4bo6[Wednesday 29 September 2010] [04:20:42] Quit iFire has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [04:59:07] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [05:25:16] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 29 September 2010] [05:39:03] Join omarkj has joined this channel (
~omarkj@81.15.34.139).
[Wednesday 29 September 2010] [05:43:31] Quit keffo has left this server (Ping timeout: 252 seconds).
[Wednesday 29 September 2010] [06:04:01] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [06:17:12] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [06:34:19] Quit iFire` has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [06:39:52] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 29 September 2010] [06:50:58] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [07:34:27] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Wednesday 29 September 2010] [07:35:42] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [08:21:00] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [08:35:11] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [10:00:35] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 29 September 2010] [10:04:42] Quit bgranger has left this server (Ping timeout: 265 seconds).
[Wednesday 29 September 2010] [10:07:06] <llasm> hi !!
[Wednesday 29 September 2010] [10:10:12] <sustrik> llasm: hi
[Wednesday 29 September 2010] [10:49:56] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [10:51:08] <llasm> how to install zmq*
[Wednesday 29 September 2010] [10:51:15] <llasm> i need a tuto
[Wednesday 29 September 2010] [10:53:54] <keffo> define ‘install’?
[Wednesday 29 September 2010] [10:55:48] <pieterh> please tell me if anything is missing or unclear on the instructions there
[Wednesday 29 September 2010] [10:58:50] Quit bgranger has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [11:00:23] Quit eintr has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [11:19:32] Quit bgranger has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [11:21:14] Quit jsimmons has left this server (Ping timeout: 255 seconds).
[Wednesday 29 September 2010] [11:27:49] <llasm> pieterh: i follow this instructions
[Wednesday 29 September 2010] [11:28:20] <llasm> but when try python -c “import zmq”
[Wednesday 29 September 2010] [11:28:40] <llasm> i receive cannot import name _zmq
[Wednesday 29 September 2010] [11:29:29] <pieterh> llasm: ok, you need to also build the Python API
[Wednesday 29 September 2010] [11:30:55] <pieterh> did you install 0MQ into system libraries (‘sudo make install’)?
[Wednesday 29 September 2010] [11:31:23] <pieterh> then you have to specify the system libraries in setup.cfg before doing ‘python setup.py install’
[Wednesday 29 September 2010] [11:31:41] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [11:32:19] <llasm> yes i use this command ‘make install’ with root user
[Wednesday 29 September 2010] [11:33:07] Quit friendshi has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [11:33:11] <pieterh> what does your setup.cfg contain?
[Wednesday 29 September 2010] [11:33:17] <llasm> but i don’t find zeromq2-dev folder in /user/local/
[Wednesday 29 September 2010] [11:34:20] <sustrik> there’s no such folder
[Wednesday 29 September 2010] [11:34:32] <sustrik> you should see /user/local/include/zmq.h for example
[Wednesday 29 September 2010] [11:35:25] <friendshi> llasm: see the section titled ‘Linux’ under ‘Building and installation’ on this page:
http://github.com/zeromq/pyzmq[Wednesday 29 September 2010] [11:37:23] <llasm> my setup.cfg contain
[Wednesday 29 September 2010] [11:37:28] <llasm> library_dirs = /usr/local/zeromq-dev/lib
[Wednesday 29 September 2010] [11:37:32] <llasm> include_dirs = /usr/local/zeromq-dev/include
[Wednesday 29 September 2010] [11:38:58] <friendshi> try this: python setup.py build_ext –rpath=/usr/local/zeromq-dev/lib –inplace
[Wednesday 29 September 2010] [11:39:51] <friendshi> if /usr/local/zeromq-dev/lib is where you actually installed zeromq library
[Wednesday 29 September 2010] [11:40:24] <pieterh> llasm: this is a common mistake, the example setup.cfg is confusing
[Wednesday 29 September 2010] [11:40:45] <pieterh> edit it so that the paths are /usr/local/lib and /usr/local/include
[Wednesday 29 September 2010] [11:42:40] <sustrik> where’s the example?
[Wednesday 29 September 2010] [11:43:22] <pieterh> setup.cfg.template
[Wednesday 29 September 2010] [11:43:30] <sustrik> pyzmq project?
[Wednesday 29 September 2010] [11:43:35] <pieterh> yes
[Wednesday 29 September 2010] [11:43:49] <pieterh> i’m making a quickstarter for Python on the www.zeromq.com download page
[Wednesday 29 September 2010] [11:44:00] <sustrik> ok, let me create a ticket asking to change the example
[Wednesday 29 September 2010] [11:44:09] <pieterh> ack
[Wednesday 29 September 2010] [11:44:16] <llasm> i think that the installation is ok. so i’ve this error when i try : python -c ‘import zmq’
[Wednesday 29 September 2010] [11:44:17] <llasm> ImportError: libzmq.so.0: cannot open shared object file: No such file or directory
[Wednesday 29 September 2010] [11:44:20] <pieterh> also the page should not IMO tell the user to get the zeromq git
[Wednesday 29 September 2010] [11:44:21] <llasm> is normal?
[Wednesday 29 September 2010] [11:44:35] <pieterh> llasm: after zeromq install, ‘sudo ldconfig’
[Wednesday 29 September 2010] [11:45:32] <friendshi> thats the error i was getting before doing the build_ext part from the documentation
[Wednesday 29 September 2010] [11:46:29] <pieterh> friendshi, ah, thanks
[Wednesday 29 September 2010] [11:52:44] <llasm> i excute ‘ldconfig’ but i receive the same error
[Wednesday 29 September 2010] [11:53:31] <sustrik> is there zmq.h in /usr/local/include?
[Wednesday 29 September 2010] [11:55:14] <llasm> yes
[Wednesday 29 September 2010] [11:55:48] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [11:55:52] <pieterh> llasm: did you run ‘sudo python setup.py install’ again?
[Wednesday 29 September 2010] [11:56:04] <llasm> yes
[Wednesday 29 September 2010] [12:00:17] * pieterh has exhausted his pyzmq knowledge
[Wednesday 29 September 2010] [12:00:39] <sustrik> how does you setup.py look like?
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> my setup.cfg contain
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> library_dirs = /usr/local/zeromq-dev/lib
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> include_dirs = /usr/local/zeromq-dev/include
[Wednesday 29 September 2010] [12:04:52] <mato> this would be a problem
[Wednesday 29 September 2010] [12:05:01] <mato> /usr/local/zeromq-dev will not be in a default LD_LIBRARY_PATH
[Wednesday 29 September 2010] [12:05:05] <mato> don’t do that :-)
[Wednesday 29 September 2010] [12:05:27] <mato> if you want things to just work then install everything in /usr/local
[Wednesday 29 September 2010] [12:05:47] <mato> or edit your ld.so.conf / LD_LIBRARY_PATH but normally you don’t want to do that
[Wednesday 29 September 2010] [12:06:41] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [12:12:18] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [12:24:58] <llasm> LD_LIBRARY_PATH is empty. it’s normal?!!
[Wednesday 29 September 2010] [12:25:19] <guido_g> yes
[Wednesday 29 September 2010] [12:26:13] <llasm> ok
[Wednesday 29 September 2010] [12:28:08] <sustrik> have you changed the setup.py?
[Wednesday 29 September 2010] [12:28:55] <sustrik> it should look like this:
[Wednesday 29 September 2010] [12:29:01] <sustrik> library_dirs = /usr/local/lib
[Wednesday 29 September 2010] [12:29:11] <sustrik> include_dirs = /usr/local/include
[Wednesday 29 September 2010] [12:29:47] <ntelford> hi guys
[Wednesday 29 September 2010] [12:30:13] <ntelford> just having a dig around the Java API and I’m wondering if there’s any support for streaming very large messages without having to buffer them in local memory first?
[Wednesday 29 September 2010] [12:30:41] <ntelford> I ask because Socket.send(byte[] msg, long flags) requires the message to be buffered as a byte array first
[Wednesday 29 September 2010] [13:08:01] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [13:24:33] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 29 September 2010] [14:03:08] Quit llasm has left this server (Quit: Quitte).
[Wednesday 29 September 2010] [14:36:38] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Wednesday 29 September 2010] [14:47:16] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 29 September 2010] [15:45:46] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [16:26:43] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [16:43:47] Quit psino has left this server (Quit: psino).
[Wednesday 29 September 2010] [16:53:04] Quit novas0x2a has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [17:31:03] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 29 September 2010] [17:45:31] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [17:45:33] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Wednesday 29 September 2010] [17:47:27] Quit eintr has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [17:48:12] Quit xla has left this server (Ping timeout: 272 seconds).
[Wednesday 29 September 2010] [18:31:43] Quit friendshi has left this server (Remote host closed the connection).
[Wednesday 29 September 2010] [19:34:41] Quit rgl has left this server (Ping timeout: 252 seconds).
[Wednesday 29 September 2010] [19:40:32] <novas0x2a> could someone confirm that I’m using zmq_poll correctly here?
http://codepad.org/iJuspYER I reliably get an uninitialized/corrupted message back out. I’m suspecting a zmq bug, but I’d like to rule out user error.
[Wednesday 29 September 2010] [19:42:22] Quit Tasser has left this server (Read error: Operation timed out).
[Wednesday 29 September 2010] [19:46:23] Quit chmod_work has left this server (Quit: POOF!).
[Wednesday 29 September 2010] [19:55:33] Quit
mw_ has left this server (Ping timeout: 245 seconds).
[Wednesday 29 September 2010] [20:30:18] Join novas0x2a1 has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [20:33:07] Quit novas0x2a has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [20:38:57] <acts_as> Does zeromq have to talk to other zeromq (or
*MQ) hosts? Or can I have zeromq talk to xyz proprietary packets?
[Wednesday 29 September 2010] [20:42:20] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 29 September 2010] [20:47:07] Quit novas0x2a1 has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [20:57:12] <acts_as> anyone here :(
[Wednesday 29 September 2010] [21:01:58] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 29 September 2010] [21:13:59] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 29 September 2010] [21:14:48] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [21:52:07] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Wednesday 29 September 2010] [22:21:10] Quit novas0x2a has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [22:48:37] Quit RobertLJ has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [23:10:58] Quit Tasser has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [23:16:18] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [23:17:43] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [23:37:12] Quit miguelito has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [00:12:15] <jasong_at_apache> I just found out about 0mq
[Thursday 30 September 2010] [00:12:25] <jasong_at_apache> i’m like in love
[Thursday 30 September 2010] [00:19:21] Quit grahamalot has left this server (Quit: grahamalot).
[Thursday 30 September 2010] [00:22:37] Quit lluad has left this server (Quit: lluad).
[Thursday 30 September 2010] [00:30:35] <Pe_Ell> heh, yeah I want to use it but I need a quick working prototype in Perl and the current drivers are blocking...
[Thursday 30 September 2010] [00:30:53] <Pe_Ell> need to look into making an AnyEvent driver but prototype comes first... but I really like the project. :)
[Thursday 30 September 2010] [00:59:22] <lestrrat> Pe_Ell: wait till zeromq 2.1
[Thursday 30 September 2010] [00:59:41] <lestrrat> I’m going to make an AnyEvent adaptor then.
[Thursday 30 September 2010] [01:00:10] <Pe_Ell> yeah, was that you I was talking to about it through email last week?
[Thursday 30 September 2010] [01:00:23] <lestrrat> Ah, duh. must be.
[Thursday 30 September 2010] [01:00:25] <Pe_Ell> I started looking at the C code but I needed to undistract myself... :)
[Thursday 30 September 2010] [01:00:29] <lestrrat> Sounded familiar.
[Thursday 30 September 2010] [01:00:56] <Pe_Ell> 0mq was a shiny object in my time of planning...
[Thursday 30 September 2010] [01:01:07] <lestrrat> :)
[Thursday 30 September 2010] [01:01:11] <Pe_Ell> so doing my initial design testing with AnyEvent::MP
[Thursday 30 September 2010] [01:01:23] <lestrrat> I see.
[Thursday 30 September 2010] [01:01:33] <Pe_Ell> but I want to switch to 0mq after I get it working
[Thursday 30 September 2010] [01:02:34] <lestrrat> I would be willing to build a prototype based on the current master, but I’m also running YAPC::Asia, so I won’t have time to hack on it until it’s over :/
[Thursday 30 September 2010] [01:02:44] <Pe_Ell> np
[Thursday 30 September 2010] [01:03:05] <Pe_Ell> I’ve got a like a week or two of my project before I can get back to it...
[Thursday 30 September 2010] [01:03:15] <lestrrat> so maybe sometime around the end of Oct... unless $dayjob catches me.
[Thursday 30 September 2010] [01:03:16] <lestrrat> righto
[Thursday 30 September 2010] [01:03:21] <Pe_Ell> from what I read look like you can use ZMQ_FD to trigger polling
[Thursday 30 September 2010] [01:03:40] <Pe_Ell> but I didn’t get terribly far into it
[Thursday 30 September 2010] [01:03:47] <lestrrat> right. you just need to pass that FD as a IO object, and register a callback using AE::io
[Thursday 30 September 2010] [01:04:14] <lestrrat> then call ZMQ’s poll function (which I need to change the API a bit...)
[Thursday 30 September 2010] [01:04:41] <Pe_Ell> yeah I was wondering if it would be that simple or not... since they mentioned that it triggers on more than just incoming events
[Thursday 30 September 2010] [01:04:45] <Pe_Ell> so you might not have any events when it triggers
[Thursday 30 September 2010] [01:05:36] <lestrrat> well, there may be caveats I’m missing, but sounds like you should be able to poll() and just have it return immediately w/o any messages.
[Thursday 30 September 2010] [01:05:40] <lestrrat> it wouldn’t matter.
[Thursday 30 September 2010] [01:05:56] <Pe_Ell> that’s what I was hoping
[Thursday 30 September 2010] [01:05:57] <lestrrat> it /would/ matter if it /didn’t/ trigger when there’s a message.
[Thursday 30 September 2010] [01:06:07] <Pe_Ell> yeah
[Thursday 30 September 2010] [01:07:03] <lestrrat> we’ll see. we might need to write a couple of wrappers around the FDs, but as long as we can decorate it like Perl IO object... we should be fine
[Thursday 30 September 2010] [01:08:10] <Pe_Ell> yeah
[Thursday 30 September 2010] [01:35:47] Quit rgl has left this server (Ping timeout: 276 seconds).
[Thursday 30 September 2010] [01:52:19] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [02:11:12] Quit sustrik has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [02:46:20] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: 03Steven McCoy 07master * r96d85b2 10/ (4 files): (log message trimmed)
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Add assertions to check for OpenPGM calls with invalid parameters.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Assertion to check that pgm_getaddrinfo is actually returning something.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Missing pgm_connect call.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Typo on TOS causing immediate abort.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Placeholder calls for timeouts whilst continuing spin loop functionality.
[Thursday 30 September 2010] [03:12:44] <CIA-20> zeromq2: * OpenPGM v5 now supports reference counting so remove init checks.
[Thursday 30 September 2010] [03:12:44] <CIA-20> zeromq2: 03Steven McCoy 07master * rd14be62 10/ (6 files): more fixes to (e)pgm transport -
http://bit.ly/9U1BOZ[Thursday 30 September 2010] [03:12:45] <CIA-20> zeromq2: 03Ilja Golshtein 07master * rec2e859 10/ (AUTHORS include/zmq.hpp): C++ binding destructors don’t throw exceptions -
http://bit.ly/9wp2gs[Thursday 30 September 2010] [03:14:20] Quit guido_g has left this server (Quit: Boom...).
[Thursday 30 September 2010] [03:16:48] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [03:38:04] Join eintr has joined this channel (
~dln@88.80.182.68).
[Thursday 30 September 2010] [04:01:51] Quit jasong_at_apache has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [04:06:19] Quit novas0x2a has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [04:06:43] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Thursday 30 September 2010] [04:21:50] Part jasong_at_apac-1 has left this channel.
[Thursday 30 September 2010] [05:04:03] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [05:13:16] Quit mytrile has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [05:28:20] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 30 September 2010] [05:33:55] <CIA-20> zeromq2: 03Steven McCoy 07master * reaa925e 10/ src/pgm_socket.cpp : FreeBSD doesn’t define MSG_ERRQUEUE – now it’s defined in 0MQ -
http://bit.ly/9p2Kny[Thursday 30 September 2010] [05:35:28] <CIA-20> zeromq2: 03Martin Sustrik 07master * r7881ba1 10/ src/pgm_socket.cpp : minor whitespace fix -
http://bit.ly/dm6tfs[Thursday 30 September 2010] [05:38:03] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [05:59:21] Quit omarkj has left this server (Client Quit).
[Thursday 30 September 2010] [06:12:11] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Thursday 30 September 2010] [06:12:26] <
js__> hi
[Thursday 30 September 2010] [06:13:23] <
js__> im having problem with the importation of the dll into visual studio 2010
[Thursday 30 September 2010] [06:14:15] <
js__> is anybody have face this trouble ?
[Thursday 30 September 2010] [06:14:43] <
js__> for the c# implementation
[Thursday 30 September 2010] [06:16:12] Quit
js__ has left this server (Client Quit).
[Thursday 30 September 2010] [07:32:28] Quit sbahra has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [07:43:00] Part jasong_at_apache has left this channel.
[Thursday 30 September 2010] [07:45:54] Quit unenana has left this server (Client Quit).
[Thursday 30 September 2010] [07:48:58] Part jasong_at_apache has left this channel.
[Thursday 30 September 2010] [08:04:05] <CIA-20> zeromq2: 03Martin Sustrik 07master * r98fa2fa 10/ src/xrep.cpp : fix in XREP’s fair queueing mechanism -
http://bit.ly/8Zh1WS[Thursday 30 September 2010] [08:05:39] Quit pasha_bastov has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [08:05:55] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [08:05:57] Nick
pasha_bastov_ is now known as pasha_bastov.
[Thursday 30 September 2010] [08:06:23] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [08:08:00] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 30 September 2010] [08:08:13] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [08:12:04] Quit pasha_bastov has left this server (Ping timeout: 272 seconds).
[Thursday 30 September 2010] [08:24:48] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 30 September 2010] [08:41:47] <mrshpot> hello, mato
[Thursday 30 September 2010] [08:55:07] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [08:59:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [09:10:58] <CIA-20> zeromq2: 03Max Wolf 07maint * r79aea5f 10/ (AUTHORS src/xrep.cpp): zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/9EVmvj[Thursday 30 September 2010] [09:13:15] <CIA-20> zeromq2: 03Max Wolf 07master * r79aea5f 10/ (AUTHORS src/xrep.cpp): zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/9EVmvj[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: 03Martin Sustrik 07master * rac9b05c 10/ (AUTHORS src/xrep.cpp):
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: Merge branch ‘maint’
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: * maint:
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/dbjJBL[Thursday 30 September 2010] [09:16:23] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: 03Martin Lucina 07master * r8f9080e 10/ src/zmq.cpp :
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: zmq_poll(): Fix busy-loop if timeout is zero
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: Fix a case where zmq_poll() (poll-based version) could go off into a busy-loop
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: if no revents are returned and the timeout passed in is zero. -
http://bit.ly/dq8ZsW[Thursday 30 September 2010] [09:35:58] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [09:53:03] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [09:59:34] <mato> sustrik: are you there?
[Thursday 30 September 2010] [09:59:41] <sustrik> yes
[Thursday 30 September 2010] [09:59:50] <mato> i get an occasional assertion on master
[Thursday 30 September 2010] [09:59:56] <mato> no idea what causes it
[Thursday 30 September 2010] [10:00:02] <sustrik> what’s the assertion?
[Thursday 30 September 2010] [10:00:03] <mato> Assertion failed:
inpipe_ &&
outpipe_ (xrep.cpp:53)
[Thursday 30 September 2010] [10:01:29] <sustrik> that should not happen
[Thursday 30 September 2010] [10:01:32] <mato> will try to get a core dump
[Thursday 30 September 2010] [10:01:33] <sustrik> can we reproduce it?
[Thursday 30 September 2010] [10:01:37] <mato> it’s very random
[Thursday 30 September 2010] [10:01:40] <sustrik> hm
[Thursday 30 September 2010] [10:08:24] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [10:15:47] <mato> sustrik: ok, i have a core file... it comes out of a case with close()
[Thursday 30 September 2010] [10:15:52] <mato> sustrik: so it’s related to socket shutdown
[Thursday 30 September 2010] [10:15:59] <sustrik> ack
[Thursday 30 September 2010] [10:16:08] <sustrik> can i see the backtrace?
[Thursday 30 September 2010] [11:01:41] Quit eintr has left this server (Quit: Leaving.).
[Thursday 30 September 2010] [11:03:00] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Thursday 30 September 2010] [11:39:58] <mikko> ZMQ_TYPE support added to php bindings
[Thursday 30 September 2010] [11:40:08] <mikko> i like the changed semantics with zmq_close
[Thursday 30 September 2010] [11:40:22] <mikko> with languages like php the ‘old’ semantics cause a lot of issues
[Thursday 30 September 2010] [11:40:40] <sustrik> it caused lot of problems to everone
[Thursday 30 September 2010] [11:41:13] <sustrik> but even the new solution is not perfect, zmq_term may now block for prolonged periods of time
[Thursday 30 September 2010] [11:41:32] Quit lluad has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [11:41:49] <sustrik> the only real solution is to have 0mq in kernel space
[Thursday 30 September 2010] [11:41:52] <mato> it still needs discussion and fixing, /me would love to write all the close semantics down properly but is bogged down in work for a client
[Thursday 30 September 2010] [11:42:10] <mato> sustrik: that solution is years away, hence not a solution :)
[Thursday 30 September 2010] [11:42:34] <sustrik> sure
[Thursday 30 September 2010] [11:42:58] <mikko> zmq_close with timeout would be nice i guess
[Thursday 30 September 2010] [11:43:11] <mikko> as intermediate solution
[Thursday 30 September 2010] [11:43:16] <sustrik> yeah, SO_LINGER has to be added
[Thursday 30 September 2010] [11:43:24] <sustrik> it’s on TODO list
[Thursday 30 September 2010] [11:43:31] <mikko> im one step behind
[Thursday 30 September 2010] [11:43:38] <mikko> been on holidays in sunny istanbul
[Thursday 30 September 2010] [11:43:44] <mikko> mainly focusing on kebab and raki
[Thursday 30 September 2010] [11:43:44] <sustrik> lucky you
[Thursday 30 September 2010] [11:44:12] <mikko> how did the meeting go in london?
[Thursday 30 September 2010] [11:44:28] <sustrik> there was a tube strike exactly that day :)
[Thursday 30 September 2010] [11:44:41] <sustrik> the only one who managed to get there was jon dyte
[Thursday 30 September 2010] [11:45:07] <sustrik> pieter was more lucky with chicago meetup
[Thursday 30 September 2010] [11:45:13] <mikko> do you come to london often?
[Thursday 30 September 2010] [11:45:20] <sustrik> depends
[Thursday 30 September 2010] [11:45:35] <sustrik> you’ve been in amsterdam last time, right?
[Thursday 30 September 2010] [11:45:39] <mikko> yeah
[Thursday 30 September 2010] [11:45:52] <sustrik> gonzalo just proposed another meetup in amsterdam
[Thursday 30 September 2010] [11:45:59] <sustrik> won’t you be around by chance?
[Thursday 30 September 2010] [11:46:00] <mikko> yeah, i saw the email
[Thursday 30 September 2010] [11:46:08] <mikko> it looks like im in utrecht on 18th onwards
[Thursday 30 September 2010] [11:46:15] <mikko> which is short way from amsterdam
[Thursday 30 September 2010] [11:46:21] <mikko> s/on/from/
[Thursday 30 September 2010] [11:46:29] <sustrik> nice
[Thursday 30 September 2010] [11:46:34] <mikko> that’s week early though
[Thursday 30 September 2010] [11:46:53] <mikko> it’s either one week or two week job
[Thursday 30 September 2010] [11:47:02] <sustrik> i see
[Thursday 30 September 2010] [11:47:32] <sustrik> well, let’s see whether some other folks are interested in the meetup
[Thursday 30 September 2010] [11:47:59] <mikko> amsterdam is pretty easy to get from most of the europe
[Thursday 30 September 2010] [11:48:05] <mikko> 45 mins flight from here
[Thursday 30 September 2010] [11:48:19] <sustrik> ack
[Thursday 30 September 2010] [11:50:47] <mikko> i would be keen on meet-up if an opportunity comes up
[Thursday 30 September 2010] [11:51:20] <mikko> hmmm, i get a quota of conference days to spend from work
[Thursday 30 September 2010] [11:51:30] <mikko> are you talking in any upcoming event?
[Thursday 30 September 2010] [11:53:12] <sustrik> nope, it’s just a beer event
[Thursday 30 September 2010] [11:54:08] <sustrik> i thought that if 5-6 people would come there it would be worth of spending a day in amsterdam
[Thursday 30 September 2010] [11:54:17] <sustrik> but it’s just social
[Thursday 30 September 2010] [11:55:22] <sustrik> btw, one think we’ve talked about with jon dyte is whether it wouldn’t make sense to organise a small 0mq meetup/conference
[Thursday 30 September 2010] [11:55:58] Quit keffo has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [11:56:09] <mikko> that would be nice
[Thursday 30 September 2010] [11:56:31] <mikko> varnish people do those occasionally
[Thursday 30 September 2010] [11:56:32] <sustrik> it’s purely theoretical so far
[Thursday 30 September 2010] [11:56:40] <mikko> they also brainstorm on features etc
[Thursday 30 September 2010] [11:56:57] <sustrik> ack
[Thursday 30 September 2010] [11:57:05] <sustrik> the logistics is obvious problem
[Thursday 30 September 2010] [11:57:14] <sustrik> people are all around the globe
[Thursday 30 September 2010] [11:57:28] <mikko> that kind of thing
[Thursday 30 September 2010] [11:57:53] <sustrik> yes, something like that
[Thursday 30 September 2010] [11:57:55] <mikko> maybe we could try to organise it around a larger conference
[Thursday 30 September 2010] [11:58:02] <sustrik> one option would be to hold it after fossdem
[Thursday 30 September 2010] [11:58:05] <sustrik> in brussels
[Thursday 30 September 2010] [11:58:11] <sustrik> that’s a really big conference
[Thursday 30 September 2010] [11:58:24] <sustrik> some 2000 attendeed or so
[Thursday 30 September 2010] [11:58:46] <sustrik> held jan/feb iirc
[Thursday 30 September 2010] [11:59:29] <mikko> feb it seems
[Thursday 30 September 2010] [11:59:38] <mikko> they give rooms to distribution miniconferences
[Thursday 30 September 2010] [11:59:42] <mikko> conferences*
[Thursday 30 September 2010] [11:59:54] <mikko> there was no typo actually
[Thursday 30 September 2010] [12:00:05] <mikko> anyway, it seems like a sensible idea
[Thursday 30 September 2010] [12:00:22] <mikko> that way people can attend fossdem and probably get their companies to pay for flights / accomodation
[Thursday 30 September 2010] [12:00:34] <sustrik> there are imatix offices in brussels, so it would be easy
[Thursday 30 September 2010] [12:00:42] <mikko> We hereby invite ideas and proposals for main speakers, and proposals for developer rooms. The call for stands and lightning talks will be done after announcing the accepted devrooms.
[Thursday 30 September 2010] [12:00:54] <mikko> they also provide dev rooms
[Thursday 30 September 2010] [12:01:19] <sustrik> but pieter has a fridge with beer at imatix office :)
[Thursday 30 September 2010] [12:01:37] <mikko> sounds like a plan
[Thursday 30 September 2010] [12:01:45] <mikko> mussels in brussels
[Thursday 30 September 2010] [12:01:49] <sustrik> right
[Thursday 30 September 2010] [12:02:11] <mikko> maybe fire the idea to mailing list and see if people pick up?
[Thursday 30 September 2010] [12:03:00] <omarkj> Oh, a reason to go to Brussels?
[Thursday 30 September 2010] [12:05:23] <mikko> it would be nice to brainstorm and bounce around ideas on how people use zeromq
[Thursday 30 September 2010] [12:07:30] <sustrik> yes, if people are intersted we can arrange that
[Thursday 30 September 2010] [12:13:09] <mikko> seems like pfsense uses the zmq php bindings
[Thursday 30 September 2010] [12:13:39] Quit mrshpot has left this server (Quit: Leaving).
[Thursday 30 September 2010] [12:13:50] <sustrik> what’s that?
[Thursday 30 September 2010] [12:13:55] Quit vagn has left this server (Quit: Ex-Chat).
[Thursday 30 September 2010] [12:13:59] <mikko> it’s a firewall / captive portal
[Thursday 30 September 2010] [12:14:15] <sustrik> interesting
[Thursday 30 September 2010] [12:14:26] <mikko> “pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router.”
[Thursday 30 September 2010] [12:14:42] <mikko> it seems they use zeromq to distribute config changes
[Thursday 30 September 2010] [12:14:42] <sustrik> any idea what do they use it for?
[Thursday 30 September 2010] [12:14:47] <sustrik> i see
[Thursday 30 September 2010] [12:15:15] <sustrik> actually, i would like to see 0MQ devices running on routers one day
[Thursday 30 September 2010] [12:15:17] <mikko> i guess that enables centralised management for firewall clusters
[Thursday 30 September 2010] [12:15:33] <sustrik> but we are not that fat yet
[Thursday 30 September 2010] [12:15:38] <sustrik> far*
[Thursday 30 September 2010] [12:16:40] <sustrik> the idea is that a device sits at the edge of the network (in router) and redistributes messages to the local network
[Thursday 30 September 2010] [12:17:41] <mikko> i think some of the concepts in zeromq are hard to grasp
[Thursday 30 September 2010] [12:17:44] <mikko> for some people
[Thursday 30 September 2010] [12:17:54] <mikko> well, not zeromq specific issue
[Thursday 30 September 2010] [12:18:01] <mikko> more like messaging oriented approach
[Thursday 30 September 2010] [12:18:18] <mikko> and some people seem to insist on ‘persistent’ messaging
[Thursday 30 September 2010] [12:18:44] <sustrik> yes, i am aware of that
[Thursday 30 September 2010] [12:18:51] <sustrik> we need to educate the crowd
[Thursday 30 September 2010] [12:56:19] <jasong_at_apache> one my main dev machine which is OSX (production is ubuntu) i’m getting: ./configure: line 15263: syntax error near unexpected token
`newline’
[Thursday 30 September 2010] [12:56:20] <jasong_at_apache> ./configure: line 15263: ` PKG_CHECK_MODULES(‘
[Thursday 30 September 2010] [12:56:48] <jasong_at_apache> this is optionless ./autogen.sh , ./configure (stock as is)
[Thursday 30 September 2010] [12:58:28] <jasong_at_apache> update: i just found the doc page....
[Thursday 30 September 2010] [12:59:03] <cremes> jasong_at_apache: make sure you have libtool installed on your machine
[Thursday 30 September 2010] [12:59:17] <cremes> if you use macports, do a “port install libtool” and try again
[Thursday 30 September 2010] [12:59:35] <jasong_at_apache> i do, will try! thx cremes
[Thursday 30 September 2010] [13:05:20] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [13:25:05] Quit lluad has left this server (Client Quit).
[Thursday 30 September 2010] [14:00:58] Quit friendshi has left this server (Quit: friendshi).
[Thursday 30 September 2010] [14:16:11] <jasong_at_apache> another issue, now with jzmq
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> libtool: compile: g++ -DHAVE_CONFIG_H -I. -D_REENTRANT -D_THREAD_SAFE -I/Library/Java/Home/include -Wall -g -O2 -MT libjzmq_la-Socket.lo -MD -MP -MF .deps/libjzmq_la-Socket.Tpo -c Socket.cpp -fno-common -DPIC -o .libs/libjzmq_la-Socket.o
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> Socket.cpp: In function ‘jlong Java_org_zeromq_ZMQ_00024Socket_getLongSockopt(JNIEnv*, _jobject*, jint)’:
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> Socket.cpp:103: error: ‘ZMQ_TYPE’ was not declared in this scope
[Thursday 30 September 2010] [14:21:09] <cremes> jasong_at_apache: i don’t think the “official” bindings have been updated to work with 2.1 yet
[Thursday 30 September 2010] [14:21:25] <cremes> if you are just starting to experiment, you might want to stick with 2.0.9
[Thursday 30 September 2010] [14:21:58] <jasong_at_apache> ah
[Thursday 30 September 2010] [14:23:02] <jasong_at_apache> sorry I’m coming across as a total noob, I am monitoring several builds figure i’d just hit you guys up for the obvious
[Thursday 30 September 2010] [14:29:11] Quit omarkj has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [14:30:20] Nick n is now known as Guest28979.
[Thursday 30 September 2010] [14:31:38] <Guest28979> I want to setup a pub-sub system where a client sends a message to a central server that does some authentication and error checking and then publishes that message to all clients.
[Thursday 30 September 2010] [14:32:11] <omarkj> Okay.
[Thursday 30 September 2010] [14:32:24] <Guest28979> I’m doing a request/response pair to send the message to the server, and pub/sub to send out the messages, and it’s working beautifully, except...
[Thursday 30 September 2010] [14:33:00] <Guest28979> These are dynamic hosts, and on ocassion, the server will die and move to a different machine.
[Thursday 30 September 2010] [14:33:42] <Guest28979> Is there an elegant way to do this, or do I need to do non-blocking recv’s with timeouts, and a heartbeat on the PUB/SUB side of things?
[Thursday 30 September 2010] [14:35:54] <Peaceful> Is this a common problem? I’m trying to install zmq and pyzmq for the first time on my OS X 10.6 box, and it looked like it worked, but I get: ImportError: cannot import name _zmq –> when I do “import zmq”
[Thursday 30 September 2010] [14:36:49] <Peaceful> I just downloaded 2.0.9, untarred it, ran ./configure, make, make install ...
[Thursday 30 September 2010] [14:37:05] <Guest28979> Peaceful: How did you install pyzmq?
[Thursday 30 September 2010] [14:37:13] <Guest28979> easy_install?
[Thursday 30 September 2010] [14:37:25] <Peaceful> then downloaded pyzmq (from git), copied setup.cfg (which looked correct), and did ‘sudo python setup.py install’
[Thursday 30 September 2010] [14:37:57] <Guest28979> What was the output of that?
[Thursday 30 September 2010] [14:40:06] * Peaceful is gathering output for pastebin
[Thursday 30 September 2010] [14:45:30] <Peaceful>
http://pastebin.com/8ct0gsif <– and here’s the full output from my ”./configure”, “make”, “sudo make install”
[Thursday 30 September 2010] [14:45:55] <Peaceful> There’s warnings, but it appeared that zmq and pyzmq both successfully installed...but doesn’t work.
[Thursday 30 September 2010] [14:46:44] <guido_g> might be a path issue
[Thursday 30 September 2010] [14:47:10] <Guest28979> I’m hardly a Mac guy, so I’m stabbing in the dark here...
[Thursday 30 September 2010] [14:47:20] <guido_g> dito.
[Thursday 30 September 2010] [14:48:17] <Peaceful> Hmm. There were no mac-specific install instructions for zmq...I checked “INSTALL”. I wonder if zmq itself works okay...
[Thursday 30 September 2010] [14:48:53] <Guest28979> Do you have a directory that looks something like ‘/Library/Python/2.6/site-packages/pyzmq-2.0.9dev-py2.6.egg’?
[Thursday 30 September 2010] [14:49:24] <Peaceful> no, but...
[Thursday 30 September 2010] [14:49:43] <Peaceful> there is this, which is close:
[Thursday 30 September 2010] [14:49:45] <Peaceful> ls /Library/Python/2.6/site-packages/zmq/
[Thursday 30 September 2010] [14:49:49] <Peaceful> __init__.py __init__.pyc _zmq.pxd _zmq.so allocate.pxd buffers.pxd devices.so eventloop log tests
[Thursday 30 September 2010] [14:49:52] <Guest28979> It’s sticking an egg.info file into /Library/Python/2.6/site-package, but where is the egg itself?
[Thursday 30 September 2010] [14:50:05] <Guest28979> Ah, OK.
[Thursday 30 September 2010] [14:50:15] <Guest28979> What’s the mac equivalent of ldconfig?
[Thursday 30 September 2010] [14:50:22] <Peaceful> There’s this: /Library/Python/2.6/site-packages/pyzmq-2.0.9dev-py2.6.egg-info
[Thursday 30 September 2010] [14:50:23] <Guest28979> Try sudo ldconfig
[Thursday 30 September 2010] [14:50:42] <Peaceful> hey!
[Thursday 30 September 2010] [14:50:52] <Peaceful> before I tried that, I thought, maybe I should try a new terminal
[Thursday 30 September 2010] [14:50:58] <Peaceful> opening up a new terminal window worked
[Thursday 30 September 2010] [14:51:02] <Guest28979> Yeah, and the new one has the path.
[Thursday 30 September 2010] [14:51:06] <Peaceful> I didn’t run ldconfig
[Thursday 30 September 2010] [14:51:16] <Guest28979> Or the python path is rebuilt, or what-have-you.
[Thursday 30 September 2010] [14:51:22] <Peaceful> ah, I bet “hash -r” would have fixed it in my original one...
[Thursday 30 September 2010] [14:51:32] <Peaceful> er, nope
[Thursday 30 September 2010] [14:51:39] <Peaceful> okay, I’ll just stick with the new window :)
[Thursday 30 September 2010] [14:51:41] <Guest28979> guido_g was right, it was a path issue.
[Thursday 30 September 2010] [14:51:49] <guido_g> YES!
[Thursday 30 September 2010] [14:51:54] <Guest28979> +1
[Thursday 30 September 2010] [14:51:56] <guido_g> where’S the cookie? :)
[Thursday 30 September 2010] [14:51:58] <Peaceful> guido_g++
[Thursday 30 September 2010] [14:52:05] <Peaceful> ^– cookie
[Thursday 30 September 2010] [14:52:23] * Peaceful hands guido_g his cookie and says “good boy!”
[Thursday 30 September 2010] [14:52:28] <Peaceful> thanks!
[Thursday 30 September 2010] [14:52:32] Nick Guest28979 is now known as nedjl.
[Thursday 30 September 2010] [14:52:46] <nedjl> Let’s see if nickserv lets me have this one.
[Thursday 30 September 2010] [14:52:51] <rgl> how good is zeromq on windows?
[Thursday 30 September 2010] [14:52:52] <Peaceful> Now I’ll go see if I can actually get zmq working. I only discovered its existence last night.
[Thursday 30 September 2010] [14:52:52] * guido_g eats cookie
[Thursday 30 September 2010] [14:53:20] * Peaceful heads off to explore zmq...and eat lunch
[Thursday 30 September 2010] [14:53:22] Quit Peaceful has left this server (Quit: Peaceful).
[Thursday 30 September 2010] [14:53:32] <mikko> rgl: should be fine
[Thursday 30 September 2010] [14:53:33] <nedjl> Peaceful: There’s a “It’s not working” workflow chart that’s very good. Most of my experience with it has involved it just working.
[Thursday 30 September 2010] [14:53:46] <mikko> rgl: as far as i know, cant really vouch for it as i dont use windows
[Thursday 30 September 2010] [14:54:20] <rgl> are there any official windows binaries?
[Thursday 30 September 2010] [14:54:26] * rgl didn’t found them on the homepage
[Thursday 30 September 2010] [14:54:53] <nedjl> I figured out a better way of asking my question – is there a way to get an error when an endpoint goes down? I don’t want zeromq to hide me from this, I need to redo some service discovery to find out where I should connect to instead.
[Thursday 30 September 2010] [14:55:19] <mikko> rgl: you should be able to build easily
[Thursday 30 September 2010] [14:55:38] <rgl> mikko, ok. will do. thx! :)
[Thursday 30 September 2010] [14:55:41] <mikko> rgl: are you looking to use one of the language bindings?
[Thursday 30 September 2010] [14:55:47] <rgl> mikko, C#
[Thursday 30 September 2010] [14:56:19] <guido_g> nedjl: nope, mq is notorously underprovisioned with this kind of stuff
[Thursday 30 September 2010] [14:57:10] <guido_g> it _might_ be that you get a notification through the logging interface in the future, the distant future...
[Thursday 30 September 2010] [14:57:42] <mikko> nedjl: at the moment you would probably implement some sort of heartbeat
[Thursday 30 September 2010] [14:58:21] <mikko> it’s getting busy in here, 66 people
[Thursday 30 September 2010] [14:59:10] <guido_g> complicated by the fact that you don’t know which connection is alive and that there is no way to use a specific underlying connection
[Thursday 30 September 2010] [15:00:25] <nedjl> mikko, guido_g: OK. My current plan to figure out if the sub/pub socket is still alive to send out a heartbeat every few seconds, and kill off all the clients if there is no response.
[Thursday 30 September 2010] [15:00:58] <guido_g> if you have use-cases involving this kind of information, please take the time to mention it on the mailing list
[Thursday 30 September 2010] [15:01:12] <nedjl> The req/res socket I’m polling on the response to a req, and if I don’t get anything after a reasonable amount of time, I redo the service discovery and try again with the new info.
[Thursday 30 September 2010] [15:02:09] Quit omarkj has left this server (Quit: omarkj).
[Thursday 30 September 2010] [15:02:20] <mikko> indeed the hearbeat is problematic
[Thursday 30 September 2010] [15:02:31] <nedjl> I’ve got a server that validates messages indicating events, and then spreads em around to anyone interested. That server will go down from time to time, and I want all the clients to notice and redo the service discovery if appropriate.
[Thursday 30 September 2010] [15:02:38] <mikko> especially in scenarios where for example a specific port is blocked accidentally on a firewall
[Thursday 30 September 2010] [15:02:53] <mikko> as the hearbeat might be going through a different port than main connection
[Thursday 30 September 2010] [15:03:31] <nedjl> These are actually a cluster of EC2 servers, and I’m sending out notifications about changes in the cluster’s state using this mechanism.
[Thursday 30 September 2010] [15:04:37] <mikko> nedjl: and you dont know the peers addresses beforehand?
[Thursday 30 September 2010] [15:04:47] <nedjl> I can’t use multicast, even though it is probably the right solution, because EC2 doesn’t support multicast without some gross tunneling hacks.
[Thursday 30 September 2010] [15:05:06] <mikko> nedjl: doesnt pgm encapsulate the multicast into tcp?
[Thursday 30 September 2010] [15:05:15] <guido_g> no
[Thursday 30 September 2010] [15:05:28] <mikko> even with tcp transport?
[Thursday 30 September 2010] [15:05:34] <nedjl> Machines will get added, machines will get killed. If the machine that gets killed is the one with the server, it’ll get started up somewhere else, and everybody will just do a lookup to find the new location.
[Thursday 30 September 2010] [15:05:36] <guido_g> no
[Thursday 30 September 2010] [15:06:06] <mikko> i might’ve misunderstood pgm in that case
[Thursday 30 September 2010] [15:07:05] <guido_g> pgm is working on ip level, ephm encapsulates it in udp
[Thursday 30 September 2010] [15:07:07] <mikko> time to read rfc 3028
[Thursday 30 September 2010] [15:07:11] <guido_g>
*epgm
[Thursday 30 September 2010] [15:07:18] <mikko> ok
[Thursday 30 September 2010] [15:07:26] <mikko> i thought you could encapsulate inside tcp as well
[Thursday 30 September 2010] [15:07:45] <guido_g> you can’t do real multicast on tcp
[Thursday 30 September 2010] [15:08:18] <nedjl> mikko: 1.1 of RFC 3028 says ” PGM runs over a datagram multicast protocol such as IP multicast [5]. “
[Thursday 30 September 2010] [15:08:34] <nedjl> Curses.
[Thursday 30 September 2010] [15:08:38] <mikko> reading 3028 now
[Thursday 30 September 2010] [15:09:39] <mikko> i guess using tcp wouldnt make much sense indeed
[Thursday 30 September 2010] [15:10:08] <mikko> does epgm work in EC2?
[Thursday 30 September 2010] [15:10:18] <nedjl> So it seems inevitable that I’m looking at lots of NOBLOCK and timeouts.
[Thursday 30 September 2010] [15:10:26] <guido_g> no, because you said multicast doesn’t work
[Thursday 30 September 2010] [15:10:27] <nedjl> Lemme lok.
[Thursday 30 September 2010] [15:10:48] <mikko> guido_g: i don’t know anything about EC2
[Thursday 30 September 2010] [15:10:57] * guido_g knows even less
[Thursday 30 September 2010] [15:11:05] <mikko> but i would imagine there are use-cases for multicast in ‘cloud’
[Thursday 30 September 2010] [15:11:10] <guido_g> <nedjl> I can’t use multicast, even though it is probably the right solution, because EC2 doesn’t support multicast without some gross tunneling hacks.
[Thursday 30 September 2010] [15:11:41] <guido_g> mikko: definitely, but it’s also not easy and might even kill the network
[Thursday 30 September 2010] [15:11:45] <mikko> it might be hard to organize safely
[Thursday 30 September 2010] [15:11:59] <guido_g> so it is wise to disable it
[Thursday 30 September 2010] [15:12:33] <mikko> what’s the current best approach to heartbeats with zmq?
[Thursday 30 September 2010] [15:12:40] <mikko> use a separate oob socket for heartbeat?
[Thursday 30 September 2010] [15:12:50] <nedjl> mikko: That’s probably the question I should be asking.
[Thursday 30 September 2010] [15:12:53] <guido_g> i don’t think that there is a an easy and useful way for real multicast on these sold virtual machines
[Thursday 30 September 2010] [15:13:23] <guido_g> if you use a separate socket your testing this one, not the other one
[Thursday 30 September 2010] [15:13:47] <mikko> guido_g: i would imagine it would also require end to end encryption for the multicast
[Thursday 30 September 2010] [15:14:16] <mikko> i guess it depends on how the cloud is organised, i remember some providers have private clouds with separate VLANs
[Thursday 30 September 2010] [15:14:37] <guido_g> mikko: yes, and every one would be forced to use it... no way
[Thursday 30 September 2010] [15:15:06] <guido_g> won’t work on the “machine on demand” front
[Thursday 30 September 2010] [15:15:07] <mikko> guido_g: not forced of course, but if they cared for the authenticity of the data then yes
[Thursday 30 September 2010] [15:15:29] <mikko> i guess thats almost forcing
[Thursday 30 September 2010] [15:15:56] <guido_g> otoh multicast steals bandwidth from other users, if a subscriber is in their network segment
[Thursday 30 September 2010] [15:16:05] <guido_g> so one has to deal with that too
[Thursday 30 September 2010] [15:16:31] <guido_g> i don’t think real multicast will become widely used in to vapor... oops... cloud world
[Thursday 30 September 2010] [15:16:41] <mikko> and non rate-limited multicast would be a bit scary
[Thursday 30 September 2010] [15:21:40] <jasong_at_apache> anyone here successfully compile jzmq on osx?
[Thursday 30 September 2010] [15:23:18] Quit nedjl has left this server (Quit: Page closed).
[Thursday 30 September 2010] [15:26:50] <mikko> jasong_at_apache: what is the problem you are facing?
[Thursday 30 September 2010] [15:26:55] <mikko> i can give it a spin
[Thursday 30 September 2010] [15:27:18] <jasong_at_apache> on 10.6 i’ve successfully compiled 2.0.9 (download area)
[Thursday 30 September 2010] [15:27:42] <jasong_at_apache> yet with jzmq i get: Socket.cpp: In function ‘jlong Java_org_zeromq_ZMQ_00024Socket_getLongSockopt(JNIEnv*, _jobject*, jint)’:
[Thursday 30 September 2010] [15:27:42] <jasong_at_apache> Socket.cpp:103: error: ‘ZMQ_TYPE’ was not declared in this scope
[Thursday 30 September 2010] [15:27:51] <jasong_at_apache> during make
[Thursday 30 September 2010] [15:27:53] <mikko> ZMQ_TYPE is in 2.1 only
[Thursday 30 September 2010] [15:28:19] <mikko> and i think the type is ‘int’ iirc
[Thursday 30 September 2010] [15:28:20] <jasong_at_apache> i thought i did a checkout -b from a previous commit (i need to do that again)
[Thursday 30 September 2010] [15:28:31] <jasong_at_apache> how far back do i go? there are no tags atm
[Thursday 30 September 2010] [15:28:34] <mikko> the method name suggests that its a long
[Thursday 30 September 2010] [15:28:53] <mikko> jasong_at_apache: 2.1.0 zmq i mean
[Thursday 30 September 2010] [15:28:58] <mikko> jasong_at_apache: the current github trunk
[Thursday 30 September 2010] [15:29:23] <jasong_at_apache> so it’s not worth using the binding with the maint branch?
[Thursday 30 September 2010] [15:29:29] <sustrik> the change to jzmq was committed by gonzalo today
[Thursday 30 September 2010] [15:29:49] <sustrik> so if you grab yesteday’s version it should work with 2.0.9
[Thursday 30 September 2010] [15:30:01] <jasong_at_apache> ah
[Thursday 30 September 2010] [15:30:02] <mikko> jasong_at_apache: if you edit Socket.cpp line 103
[Thursday 30 September 2010] [15:30:13] <jasong_at_apache> and remove it
[Thursday 30 September 2010] [15:30:13] <mikko> #ifdef ZMQ_TYPE around the line should help
[Thursday 30 September 2010] [15:30:23] <jasong_at_apache> and i’ll grab the previous commit
[Thursday 30 September 2010] [15:30:32] <mikko> sustrik: ZMQ_TYPE is ‘int’ ?
[Thursday 30 September 2010] [15:30:32] <sustrik> mikko: yes
[Thursday 30 September 2010] [15:30:51] <mikko> looking at this jzmq uses int64_t
[Thursday 30 September 2010] [15:31:27] <sustrik> gonzalo messed it up
[Thursday 30 September 2010] [15:31:55] <sustrik> please, do report the problem on jzmq bugtracker
[Thursday 30 September 2010] [15:33:53] <jasong_at_apache> docs want straight int
[Thursday 30 September 2010] [15:34:01] <mikko> done
[Thursday 30 September 2010] [15:34:07] <sustrik> thanks
[Thursday 30 September 2010] [15:35:28] * sustrik wonders how to make bindings work with both maint and stable without burdening maintainers with two separate branches
[Thursday 30 September 2010] [15:35:43] <sustrik> maint & master, i meant
[Thursday 30 September 2010] [15:35:53] <jasong_at_apache> maint? who needs maint!
[Thursday 30 September 2010] [15:36:00] <mikko> it’s a lot better now
[Thursday 30 September 2010] [15:36:01] <jasong_at_apache> ;-)
[Thursday 30 September 2010] [15:36:14] <mikko> as there aren’t breakages that often
[Thursday 30 September 2010] [15:36:40] <sustrik> yeah, the versioning contract helped
[Thursday 30 September 2010] [15:36:59] <mikko> are there constants for api version etc at the moment?
[Thursday 30 September 2010] [15:37:17] <sustrik> just a plain 0mq version
[Thursday 30 September 2010] [15:37:39] <sustrik> the api version should translate into library version
[Thursday 30 September 2010] [15:37:45] <sustrik> but that’s not done yet
[Thursday 30 September 2010] [15:37:51] <mikko> i guess it’s easy to create int out of that during configure phase
[Thursday 30 September 2010] [15:38:00] <mikko> something easy to compare in preprocessor macros
[Thursday 30 September 2010] [15:38:25] <mikko> something like #if ZMQ_VERSION_ID > 2001001 #else etc
[Thursday 30 September 2010] [15:38:31] <mikko> 2001001 being 2.1.1
[Thursday 30 September 2010] [15:38:33] <sustrik> mikko: good idea
[Thursday 30 September 2010] [15:41:06] <sustrik> on the other hand, some bindings (C#, lisp) don’t even include the header file
[Thursday 30 September 2010] [15:46:47] <mikko> hmm
[Thursday 30 September 2010] [15:47:02] <mikko> i reckon there still might be value for such a thing
[Thursday 30 September 2010] [15:47:12] <jasong_at_apache> everything compiled correctly (and I’m using trunk all around), also swapped int for int64 on the zmq_type
[Thursday 30 September 2010] [15:47:30] <jasong_at_apache> i meant the other way around there
[Thursday 30 September 2010] [15:49:09] <sustrik> goodo
[Thursday 30 September 2010] [15:49:36] <sustrik> anyway, this kind of thing is a bit scary as people are using the binding is production
[Thursday 30 September 2010] [15:49:40] <sustrik> in*
[Thursday 30 September 2010] [15:49:50] <jasong_at_apache> :-)
[Thursday 30 September 2010] [15:50:04] <sustrik> mikko: yes, it will help for most bindings
[Thursday 30 September 2010] [15:50:23] <sustrik> i’ll discuss it with mato tomorrow, he’s in charge of the build system
[Thursday 30 September 2010] [15:53:37] Quit omarkj has left this server (Quit: omarkj).
[Thursday 30 September 2010] [15:55:13] <novas0x2a> i’m using zmq_poll w/ 2.0.9, REQ/REP pair, tcp or ipc proto, same context (1 process). zmq_poll never seems to notice activity. if i use inproc it works, if i use a spinloop/nonblocking-recv (no poll) it works. should i expect it to work with tcp?
[Thursday 30 September 2010] [15:56:18] <sustrik> yes
[Thursday 30 September 2010] [15:56:46] <sustrik> if you have a probelm, write a simple program showing it and send it to the mailing list
[Thursday 30 September 2010] [15:57:29] <novas0x2a> could you do a quick sanity check to make sure i’m not pebkacing?
http://codepad.org/3Pxo5V1x[Thursday 30 September 2010] [15:58:45] <novas0x2a> (if you want to compile -DUSE_POLL or -DUSE_BLOCK)
[Thursday 30 September 2010] [16:00:31] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * rd215dac 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java):
[Thursday 30 September 2010] [16:00:31] <CIA-20> jzmq: Added support for ZMQ_FD (getFD()) and ZMQ_EVENTS (getEvents()).
[Thursday 30 September 2010] [16:04:16] <sustrik> novas0x2a: the program looks ok after short inspection
[Thursday 30 September 2010] [16:04:57] <sustrik> if you want someone to actually debug it though, you have to strip as much code as possible and send it to the mailing list
[Thursday 30 September 2010] [16:10:51] <jond> novas0x2a: is this the error : raw.c:158: int main(int, char**): Assertion
`ret’ failed.?
[Thursday 30 September 2010] [16:11:39] Quit psino has left this server (Quit: psino).
[Thursday 30 September 2010] [16:11:52] <novas0x2a> jond: that’s the detection of the error, yeah (it should have printed Timeout just before that)
[Thursday 30 September 2010] [16:15:48] <jond> novas0x2a: yep but when you timeout in yr code, you return 0
[Thursday 30 September 2010] [16:15:50] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * r9ccc16b 10/ src/org/zeromq/ZMQ.java : Added support for automatically growing the size of a Poller object. -
http://bit.ly/cCKiIK[Thursday 30 September 2010] [16:17:13] <jond> novas0x2a: and asserting on that doesnt seem to be a great idea
[Thursday 30 September 2010] [16:17:46] <novas0x2a> jond: this is test code, not real code. i’ve been trying to map the conditions under which it doesn’t work, to track it down. clearly, in real code, i would handle a timeout appropriately :)
[Thursday 30 September 2010] [16:19:09] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [16:20:22] Part cremes has left this channel.
[Thursday 30 September 2010] [16:20:33] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Thursday 30 September 2010] [16:20:33] Part cremes has left this channel.
[Thursday 30 September 2010] [16:21:15] <Peaceful> okay, so I made the hello world server-client example, and it worked great until I had to kill the server, and now the server won’t bind to tcp anymore. (???)
[Thursday 30 September 2010] [16:22:12] <Peaceful> ctrl-c wouldn’t stop the server, so I ended up looking up the pid and running “kill $pid”
[Thursday 30 September 2010] [16:22:39] <Peaceful> this has got to be some common newbie problem, I figure...
[Thursday 30 September 2010] [16:23:08] <Peaceful> But usually tcp ports become available again after you’ve killed the original program that was binding the port.
[Thursday 30 September 2010] [16:23:19] <Peaceful> ...in my experience, at least
[Thursday 30 September 2010] [16:25:01] <jasong_at_apache> is there a mailing list keeping track of commits? (does github do this?)
[Thursday 30 September 2010] [16:26:29] <mikko> github does that
[Thursday 30 September 2010] [16:28:02] <jond> novas0x2a: line 63 in yr file. you redefine rc. it should read rc = zmq_poll(item, 1, timeout); and the code works
[Thursday 30 September 2010] [16:30:16] <mikko> sustrik: java handles all options as int64_t (numeric ones)
[Thursday 30 September 2010] [16:30:28] <mikko> so i guess thats not an issue
[Thursday 30 September 2010] [16:31:00] <novas0x2a> jond: oh, man, i knew it was going to be my fault somehow. sigh.
[Thursday 30 September 2010] [16:31:13] <mikko> assuming that sizeof (uint64_t) <= sizeof (int64_t) is a sane assumption
[Thursday 30 September 2010] [16:32:48] <jasong_at_apache> is that only 64bit systems?
[Thursday 30 September 2010] [16:35:43] <mikko> sizeof both is 8 on systems i have access to
[Thursday 30 September 2010] [16:59:49] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Thursday 30 September 2010] [17:17:37] Quit xla has left this server (Ping timeout: 255 seconds).
[Thursday 30 September 2010] [17:28:01] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 30 September 2010] [17:30:12] Quit rossij has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [17:32:02] Quit rgl has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [17:33:18] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Thursday 30 September 2010] [17:57:40] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [18:07:45] Quit keffo has left this server (Ping timeout: 252 seconds).
[Thursday 30 September 2010] [18:09:54] Quit jond has left this server (Quit: Leaving).
[Thursday 30 September 2010] [18:14:20] Quit friendshi has left this server (Quit: friendshi).
[Thursday 30 September 2010] [18:15:55] Quit novas0x2a has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [18:17:05] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [18:18:55] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Thursday 30 September 2010] [18:22:13] Quit plq has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [18:39:35] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 30 September 2010] [19:10:34] * Peaceful doesn’t feel the love today. Oh well, I’m sure it will start working again after a reboot.
[Thursday 30 September 2010] [19:10:44] Quit Peaceful has left this server (Quit: Peaceful).
[Thursday 30 September 2010] [19:22:37] Quit clochette has left this server (Client Quit).
[Thursday 30 September 2010] [19:23:51] Part
js__ has left this channel.
[Thursday 30 September 2010] [19:53:48] Quit _rgl has left this server (Ping timeout: 272 seconds).
[Thursday 30 September 2010] [20:03:18] Quit friendshi has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [20:37:15] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [20:51:34] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [21:07:08] Quit xla has left this server (Ping timeout: 255 seconds).
[Thursday 30 September 2010] [21:42:04] Quit bbigras has left this server (Ping timeout: 252 seconds).
[Thursday 30 September 2010] [22:01:40] Quit ChicagoBud has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [22:18:26] Join ChicagoBud has joined this channel (
~bud@208.78.39.48).
[Thursday 30 September 2010] [22:18:54] Quit jasong_at_apache has left this server (Quit: jasong_at_apache).
[Thursday 30 September 2010] [23:05:45] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [23:10:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [00:12:01] Quit poswald has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [00:48:28] Quit xrfang has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [01:08:16] Quit
xrfang_ has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:36:48] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 01 September 2010] [01:39:43] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:54:38] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [01:58:11] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47c064f 10/ (src/fq.cpp src/lb.cpp): hangup when closing socket with no pipes attached – fixed -
http://bit.ly/9iy2zX[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * raaa0761 10/ src/pair.cpp : pipe being attached to the PAIR socket during its termination process is immediately asked to terminate itself -
http://bit.ly/bwIpo1[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * rdb73c76 10/ src/pub.cpp : assert when pipe attaches to PUB socket in process of termination – fixed -
http://bit.ly/bCxg7v[Wednesday 01 September 2010] [02:01:04] <CIA-20> zeromq2: 03Martin Sustrik 07master * rce0972d 10/ (src/ctx.cpp src/ctx.hpp src/object.cpp src/object.hpp): context creates an inproc endpoint (‘inproc://log’) to distribute 0MQ’s log messages - http://bit.ly/9YrEbe
[Wednesday 01 September 2010] [02:25:20] Quit sustrik has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [02:29:02] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 01 September 2010] [02:30:13] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [03:02:01] <guido_g> howdy
[Wednesday 01 September 2010] [03:05:49] <guido_g> cool, logging...
[Wednesday 01 September 2010] [03:09:41] <guido_g> does the core itself use the logging?
[Wednesday 01 September 2010] [03:10:06] * guido_g looks at the patch
[Wednesday 01 September 2010] [03:20:18] <guido_g> hmmm
[Wednesday 01 September 2010] [03:20:23] <guido_g> doesn’t seem so
[Wednesday 01 September 2010] [03:26:11] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [03:27:08] <sustrik> only the core can use the logging
[Wednesday 01 September 2010] [03:27:27] <sustrik> however, atm it’s not logging anything
[Wednesday 01 September 2010] [03:27:46] <sustrik> logs are to be added gradually
[Wednesday 01 September 2010] [03:31:32] <guido_g> well... better then nothing :)
[Wednesday 01 September 2010] [04:50:11] <Mikael_H_Kjaer> Sustrik: I don’t have a small example that triggers the bug I’ve posted about on the mailing list.
[Wednesday 01 September 2010] [04:51:04] <sustrik> hi mikael
[Wednesday 01 September 2010] [04:51:10] <sustrik> that’s a problem
[Wednesday 01 September 2010] [04:51:30] <sustrik> from inspecting the backtrace it looks like it’s not a 0mq problem
[Wednesday 01 September 2010] [04:51:37] <Mikael_H_Kjaer> I know. I have been trying to boil it down
[Wednesday 01 September 2010] [04:51:49] <sustrik> maybe a memory overwrite?
[Wednesday 01 September 2010] [04:52:05] <sustrik> some random code overwriting the socket?
[Wednesday 01 September 2010] [04:52:07] <Mikael_H_Kjaer> Maybe. Currently I am not hitting the issue anymore
[Wednesday 01 September 2010] [04:52:55] <Mikael_H_Kjaer> But I shouldn’t have any wild pointers or memory accesses in my code. I am currently using smart pointers for everything and my cycle checker seems to think I am okay too.
[Wednesday 01 September 2010] [04:54:07] <Mikael_H_Kjaer> I haven’t valgrinded my code in a few weeks but I will soon try it with zmq added in.
[Wednesday 01 September 2010] [04:54:47] <sustrik> ok. good
[Wednesday 01 September 2010] [04:55:24] <sustrik> btw, maybe try to compile with debug into so that backtrace is more informative
[Wednesday 01 September 2010] [04:56:37] <Mikael_H_Kjaer> I do :) what additional information do you want
[Wednesday 01 September 2010] [04:58:46] <sustrik> hm
[Wednesday 01 September 2010] [04:59:14] <sustrik> can you possibly add a printf statement to socket’s destructor
[Wednesday 01 September 2010] [04:59:36] <sustrik> so that we know whether the socket was destructed or overwritten?
[Wednesday 01 September 2010] [04:59:58] <sustrik> printing out this pointer would be good so that we can correlate it with the backtrace
[Wednesday 01 September 2010] [05:00:05] <Mikael_H_Kjaer> Not a problem
[Wednesday 01 September 2010] [05:00:22] <Mikael_H_Kjaer> I will get back with more. Any other printf’s you want?
[Wednesday 01 September 2010] [05:01:53] <sustrik> not yet
[Wednesday 01 September 2010] [05:02:02] <sustrik> let’s first see whrther the socket is destructed
[Wednesday 01 September 2010] [05:06:57] Quit
miguelito_ has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [05:12:38] <Mikael_H_Kjaer> sustrik: I can’t get the error to show up any more. I believe my fix of waiting for all sends to finish before attempting a shutdown of zmq has resolved it.
[Wednesday 01 September 2010] [05:12:58] <sustrik> hm
[Wednesday 01 September 2010] [05:13:29] <sustrik> ok, so let’s leave it for now
[Wednesday 01 September 2010] [05:13:40] <sustrik> if you ever see it again, we’ll continue
[Wednesday 01 September 2010] [05:16:20] <Mikael_H_Kjaer> Agreed, I was thinking the same.
[Wednesday 01 September 2010] [05:23:43] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 01 September 2010] [05:28:05] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 01 September 2010] [05:41:42] Quit Mikael_H_Kjaer has left this server (Quit: Page closed).
[Wednesday 01 September 2010] [05:52:02] <mcxx> however, in this setup, the broker is the single point of failure
[Wednesday 01 September 2010] [05:52:54] <mcxx> can the architecture be altered somehowe? e.g. can the clients connect to multiple brokers so that when one goes down, they switch to a working one?
[Wednesday 01 September 2010] [05:54:50] <sustrik> yes, you can load balance via multiple brokers
[Wednesday 01 September 2010] [05:55:08] <sustrik> just start 2 brokers
[Wednesday 01 September 2010] [05:55:17] <sustrik> and connect all components to both of them
[Wednesday 01 September 2010] [05:55:35] <sustrik> you also have to set HWM on clients
[Wednesday 01 September 2010] [05:55:52] <sustrik> so that messages are not queued for the unavailable broker indefinitely
[Wednesday 01 September 2010] [05:56:46] Quit guido_g has left this server (Client Quit).
[Wednesday 01 September 2010] [05:57:14] <mcxx> what’s HWM? Google doesn’t show reasonable results
[Wednesday 01 September 2010] [05:57:44] <mcxx> oh, I see noe
[Wednesday 01 September 2010] [05:57:45] <mcxx> now
[Wednesday 01 September 2010] [05:59:24] Quit guido_g has left this server (Disconnected by services).
[Wednesday 01 September 2010] [05:59:30] Nick OldNugget is now known as guido_g.
[Wednesday 01 September 2010] [06:39:04] <mcxx> when I set HWM on the client and one of the brokers goes down, will the clients’ sockets find out the broker is no longer available and automatically use others?
[Wednesday 01 September 2010] [06:39:45] <guido_g> huh? what broker?
[Wednesday 01 September 2010] [06:40:12] <guido_g> you mean the pipes of the underlying connections?
[Wednesday 01 September 2010] [06:40:32] <mcxx> guido_g: it’s a follow up to my previous question
[Wednesday 01 September 2010] [06:40:38] <guido_g> as far as i know, full pipes (hwm reached) are skipped
[Wednesday 01 September 2010] [06:41:08] <guido_g> i see
[Wednesday 01 September 2010] [06:41:38] <guido_g> still, full pipes are skipped
[Wednesday 01 September 2010] [06:43:58] <mcxx> re-reading that helps :)
[Wednesday 01 September 2010] [06:46:03] <guido_g> :)
[Wednesday 01 September 2010] [07:13:31] Quit rbraley has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [07:26:08] Quit sustrik has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [07:32:01] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6a0c323 10/ (builds/msvc/libzmq/libzmq.vcproj src/ctx.cpp): MSVC build fixed -
http://bit.ly/cEMTGs[Wednesday 01 September 2010] [07:39:18] <pieterh> I’ve posted a blog that hopefully clears up the naming confusion for once and for all...
[Wednesday 01 September 2010] [07:39:27] <pieterh> anyone from Denmark here?
[Wednesday 01 September 2010] [07:40:01] Quit ivan has left this server (Quit: Coyote finally caught me).
[Wednesday 01 September 2010] [07:43:28] <guido_g> :)
[Wednesday 01 September 2010] [08:04:50] <pieterh> sustrik: random question about 2.1
[Wednesday 01 September 2010] [08:05:21] <sustrik> yes?
[Wednesday 01 September 2010] [08:07:08] <pieterh> how solid is it?
[Wednesday 01 September 2010] [08:07:18] <pieterh> should I start to be using it for the examples in the guide?
[Wednesday 01 September 2010] [08:08:01] <sustrik> it haven’t been tested extensively
[Wednesday 01 September 2010] [08:08:06] <sustrik> but the API is the same
[Wednesday 01 September 2010] [08:08:34] <sustrik> so, for the guide, it’s irrelevant which one you use
[Wednesday 01 September 2010] [08:08:46] <pieterh> there’s the ‘insert random sleep here’ which i’d like to remove
[Wednesday 01 September 2010] [08:08:55] <sustrik> ah, yes, that can go with 2.1
[Wednesday 01 September 2010] [08:09:00] <pieterh> i’ll start to test it then
[Wednesday 01 September 2010] [08:09:11] <pieterh> it’d be fun to release a 2.1.0 unstable IMO
[Wednesday 01 September 2010] [08:09:12] <sustrik> btw, there’s a test suite in the package now
[Wednesday 01 September 2010] [08:09:24] <pieterh> especially if you have stuff like sys:// in there...
[Wednesday 01 September 2010] [08:09:24] <sustrik> so if you write a test
[Wednesday 01 September 2010] [08:09:28] <sustrik> jsut put it in there
[Wednesday 01 September 2010] [08:10:05] <sustrik> well, the infrastructure is there, but no notifications are passed atm
[Wednesday 01 September 2010] [08:10:08] <pieterh> :-) i saw some emails on that but missed the original discussion
[Wednesday 01 September 2010] [08:10:15] <pieterh> great idea to have a test suite!
[Wednesday 01 September 2010] [08:10:28] <sustrik> it’s in tests subdir
[Wednesday 01 September 2010] [08:10:34] <sustrik> just put your file there
[Wednesday 01 September 2010] [08:10:39] <sustrik> add it to Makefila.am
[Wednesday 01 September 2010] [08:10:45] <sustrik> and make check will run it
[Wednesday 01 September 2010] [08:10:47] <pieterh> ack
[Wednesday 01 September 2010] [08:11:42] <guido_g> ah, while talking about tests
[Wednesday 01 September 2010] [08:12:10] <guido_g> i plan to write the basic tests for the missing socket types over the weekend
[Wednesday 01 September 2010] [08:12:30] <sustrik> missing socket types?
[Wednesday 01 September 2010] [08:12:40] <sustrik> ah, those not yet in test suite
[Wednesday 01 September 2010] [08:12:47] <guido_g> afaik only req/rep and pair are tested atm
[Wednesday 01 September 2010] [08:12:48] <sustrik> nice
[Wednesday 01 September 2010] [08:13:04] <pieterh>
ZMQ_[Wednesday 01 September 2010] [08:13:05] <guido_g> if there are any corner cases that should be tested, please leave me a note
[Wednesday 01 September 2010] [08:13:14] <pieterh> the secret socket typ
[Wednesday 01 September 2010] [08:13:18] <pieterh>
*type
[Wednesday 01 September 2010] [08:13:27] <pieterh> if you can’t spell it, you can’t use it...
[Wednesday 01 September 2010] [08:13:35] <guido_g> pieterh seems to be in a funny mood today
[Wednesday 01 September 2010] [08:13:36] <guido_g> :)
[Wednesday 01 September 2010] [08:13:56] * pieterh had too many beers with mato last night, his head is not switched on right
[Wednesday 01 September 2010] [08:13:56] <sustrik> you really got the letter or have you made it up?
[Wednesday 01 September 2010] [08:14:05] <pieterh> that’s a real letter
[Wednesday 01 September 2010] [08:14:19] <pieterh> i couldn’t invent something _that_ ... unique
[Wednesday 01 September 2010] [08:14:22] <guido_g> justifies the number of beers you had
[Wednesday 01 September 2010] [08:14:40] <pieterh> that’s why the guide made that strange reference to annoying the Danes
[Wednesday 01 September 2010] [08:15:07] <guido_g> ahhh
[Wednesday 01 September 2010] [08:15:54] <pieterh> next we’re going to get email from the mathematicians accusing us of soiling Bourbaki’s name
[Wednesday 01 September 2010] [08:16:42] <pieterh> guido_g: what language are you using for the tests?
[Wednesday 01 September 2010] [08:16:52] <guido_g> c++
[Wednesday 01 September 2010] [08:17:09] <guido_g> but they can be in any language the core understands
[Wednesday 01 September 2010] [08:17:11] <pieterh> there’s one example that would work for push/pull and pair
[Wednesday 01 September 2010] [08:17:35] <guido_g> for these sockets we have tests already
[Wednesday 01 September 2010] [08:17:38] <pieterh> the mtrelay
[Wednesday 01 September 2010] [08:17:58] <pieterh> you just said only req/rep and pair...?
[Wednesday 01 September 2010] [08:18:21] <guido_g> oh sorry
[Wednesday 01 September 2010] [08:18:33] <guido_g> parsed the pub/sub wrong
[Wednesday 01 September 2010] [08:18:46] <guido_g> push/pull
[Wednesday 01 September 2010] [08:18:48] <pieterh> agh, I realized that example in fact doesn’t use push/pull finally, i switched to using pair
[Wednesday 01 September 2010] [08:18:51] * guido_g needs new glases
[Wednesday 01 September 2010] [08:18:58] * pieterh needs a new head
[Wednesday 01 September 2010] [08:19:26] <guido_g> and i thought they do have good beer in .sk
[Wednesday 01 September 2010] [08:19:43] <pieterh> it’s ok, not real beer like we make in Belgium
[Wednesday 01 September 2010] [08:19:54] <guido_g> ah
[Wednesday 01 September 2010] [08:20:02] <pieterh> kind of halfway between real beer (belgian) and fake beer (anglo)
[Wednesday 01 September 2010] [08:20:08] <pieterh> just kidding
[Wednesday 01 September 2010] [08:20:19] <pieterh> it’s the home of beer, of course
[Wednesday 01 September 2010] [08:20:57] <pieterh> ZeroMQ is a big hit in Japan it seems
[Wednesday 01 September 2010] [08:21:15] <pieterh> RT !heavenshell: perl ZeroMQ
[Wednesday 01 September 2010] [08:21:27] <guido_g> ahh... sure
[Wednesday 01 September 2010] [08:21:42] <sustrik> there are some japanese guys working on the perl binding afaiu
[Wednesday 01 September 2010] [08:21:43] <pieterh> I think they’re complaining about the spelling... :-/
[Wednesday 01 September 2010] [08:21:51] <pieterh> my Japanese isn’t so good anymore
[Wednesday 01 September 2010] [08:22:01] Quit jond has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [08:22:13] <sustrik> btw “where 0mq goes, google cannot follow” is a nice slogan
[Wednesday 01 September 2010] [08:22:16] * pieterh wonders about MQ...
[Wednesday 01 September 2010] [08:22:28] <sustrik> wrt distributed vs. centralised services
[Wednesday 01 September 2010] [08:22:38] <sustrik> privacy etc.
[Wednesday 01 September 2010] [08:22:55] <sustrik> MQ would be read NoMQ :)
[Wednesday 01 September 2010] [08:25:48] <pieterh> Hehe, “can’t follow this”...
[Wednesday 01 September 2010] [08:27:34] <pieterh> we need to prepare an ultrasecret joke for next April 1st
[Wednesday 01 September 2010] [08:27:40] <pieterh> where we rename 0MQ to MQ
[Wednesday 01 September 2010] [08:27:54] <pieterh> get lots of people to blog and twitter about it
[Wednesday 01 September 2010] [08:27:59] <pieterh> confuse the heck out of everyone...
[Wednesday 01 September 2010] [08:31:16] <pieterh> sustrik: could you send me a .zip of the zeromq2 master (clean checkout)?
[Wednesday 01 September 2010] [08:31:29] <pieterh> i’m on a slow 2G connection and have failed 4 times to do a git pull
[Wednesday 01 September 2010] [08:31:43] <pieterh> maybe email will work better
[Wednesday 01 September 2010] [08:36:32] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Wednesday 01 September 2010] [08:36:32] Notice -NickServ- You are now identified for travlr.
[Wednesday 01 September 2010] [08:36:32] CTCP Received Version request from frigg.
[Wednesday 01 September 2010] [08:36:33] Topic The channel topic is “Welcome!”.
[Wednesday 01 September 2010] [08:36:40] Mode Channel modes: no messages from outside, topic protection
[Wednesday 01 September 2010] [08:36:40] Created This channel was created on 2010-02-05 10:44.
[Wednesday 01 September 2010] [08:39:25] <pieterh> sustrik: recv’d! thanks... git seems to have a problem working across slow links
[Wednesday 01 September 2010] [08:39:44] <pieterh> too much chatter or something... they aren’t using 0MQ....
[Wednesday 01 September 2010] [09:02:00] Quit poswald has left this server (Quit: poswald).
[Wednesday 01 September 2010] [09:10:29] <pieterh> sustrik: what’s your view of when 2.1.0 would be releasable?
[Wednesday 01 September 2010] [09:10:46] <pieterh> would it help if I stress tested it with some real applications?
[Wednesday 01 September 2010] [09:10:58] <pieterh> btw, it all build and passed the test suite fine, of course
[Wednesday 01 September 2010] [09:11:02] <pieterh>
*built
[Wednesday 01 September 2010] [09:11:48] <mikko> you can run tests in language bindings as well
[Wednesday 01 September 2010] [09:11:49] <guido_g> not a real suprise, the tests are really basic
[Wednesday 01 September 2010] [09:11:58] <mikko> they probably contain fair amount of tests
[Wednesday 01 September 2010] [09:12:58] <pieterh> gudio_g: i’d like to try breakage tests...
[Wednesday 01 September 2010] [09:13:13] <guido_g> nevertheless do we need good test coverage in the core
[Wednesday 01 September 2010] [09:13:27] <pieterh> yes, we can’t refer up the stack
[Wednesday 01 September 2010] [09:13:42] <pieterh> core can’t test using language bindings, this is up to the binding author/community
[Wednesday 01 September 2010] [09:13:55] <guido_g> pieterh: what does that mean?
[Wednesday 01 September 2010] [09:14:02] <guido_g> breakage tests
[Wednesday 01 September 2010] [09:14:17] <pieterh> well, stuff like send message, close, check that it’s delivered or not according to socket policy
[Wednesday 01 September 2010] [09:14:29] <pieterh> exceed HWM, check messages are delayed/discarded
[Wednesday 01 September 2010] [09:14:36] <pieterh> send invalid stuff, check 0MQ does not die
[Wednesday 01 September 2010] [09:14:50] <pieterh> do bad operations, invalid socket mixes, illegal sequences
[Wednesday 01 September 2010] [09:15:01] <pieterh> all the stuff normal apps do all the time... :-)
[Wednesday 01 September 2010] [09:15:23] <guido_g> ahh ok
[Wednesday 01 September 2010] [09:15:33] <guido_g> normal tests :)
[Wednesday 01 September 2010] [09:15:42] <pieterh> normal regression is of course also very useful
[Wednesday 01 September 2010] [09:15:45] <pieterh> “it still works”
[Wednesday 01 September 2010] [09:15:57] <pieterh> but “i can’t break it” is also valuable
[Wednesday 01 September 2010] [09:16:02] <guido_g> that you get with every build and make check
[Wednesday 01 September 2010] [09:16:18] <guido_g> the regression test
[Wednesday 01 September 2010] [09:16:25] <pieterh> yes
[Wednesday 01 September 2010] [09:16:37] <guido_g> ok, would be nice to have a list of tests to do
[Wednesday 01 September 2010] [09:16:49] <pieterh> it’s sometimes possible to collect test cases for issues
[Wednesday 01 September 2010] [09:16:51] <guido_g> i’ll go over it as time permits and write the test programs
[Wednesday 01 September 2010] [09:17:07] <pieterh> oh, sorry, i’m not trying to propose work... :-/
[Wednesday 01 September 2010] [09:17:20] <guido_g> it has to be done
[Wednesday 01 September 2010] [09:17:25] <pieterh> i’m more than happy to help though it would be in C not C++
[Wednesday 01 September 2010] [09:17:47] <pieterh> well, let’s start by collecting specs?
[Wednesday 01 September 2010] [09:17:53] <pieterh> sorry s/start/continue/
[Wednesday 01 September 2010] [09:18:04] <guido_g> yes
[Wednesday 01 September 2010] [09:18:19] <pieterh> perhaps this can help language binding authors make coherent test sets too
[Wednesday 01 September 2010] [09:18:29] <guido_g> to make it leight and easy to start with, a simple collection of ideas what to test would be ok i think
[Wednesday 01 September 2010] [09:18:30] <pieterh> i.e. standardized test set spec
[Wednesday 01 September 2010] [09:18:36] <pieterh> yes
[Wednesday 01 September 2010] [09:19:06] <pieterh> it would give users a tool for validating bindings
[Wednesday 01 September 2010] [09:19:16] <pieterh> “has test suite”
[Wednesday 01 September 2010] [09:20:12] <pieterh> actually it should then be written both in C++ and C
[Wednesday 01 September 2010] [09:20:25] <pieterh> i volunteer to make the C test suite then
[Wednesday 01 September 2010] [09:20:31] * pieterh opens his big mouth again
[Wednesday 01 September 2010] [09:22:25] <guido_g> :)
[Wednesday 01 September 2010] [09:22:38] <guido_g> i’d like to stick with c++ for now
[Wednesday 01 September 2010] [09:22:50] <guido_g> it’s hard enough to relearn it
[Wednesday 01 September 2010] [09:23:25] <pieterh> sure, so we will double up the test suite, that also gives people examples to compare?
[Wednesday 01 September 2010] [09:23:37] <pieterh> sustrik: do you have any problem with a C test suite in parallel to the C++ one?
[Wednesday 01 September 2010] [09:23:50] <pieterh> if it’s a stupid idea, i’m happy to retract it... :-)
[Wednesday 01 September 2010] [09:26:12] <guido_g> hehe
[Wednesday 01 September 2010] [09:26:24] * guido_g is leaving the office
[Wednesday 01 September 2010] [09:26:36] <guido_g> see you later
[Wednesday 01 September 2010] [09:27:19] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 01 September 2010] [09:29:46] <CIA-20> zeromq2: 03Martin Sustrik 07master * r651c1ad 10/ (src/ctx.cpp src/socket_base.cpp): sys transport introdced; inproc://log moved to sys://log - http://bit.ly/bmO6tM
[Wednesday 01 September 2010] [09:29:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0fe7d3c 10/ (builds/msvc/libzmq/libzmq.vcproj src/ctx.cpp): conflicts resolved -
http://bit.ly/aV1tLE[Wednesday 01 September 2010] [09:49:28] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 01 September 2010] [10:02:12] <guido_g> re
[Wednesday 01 September 2010] [10:03:13] <sustrik> pieterh: C test suite is OK
[Wednesday 01 September 2010] [10:03:31] <sustrik> but afaik the files have to have .cpp extension
[Wednesday 01 September 2010] [10:03:44] <sustrik> so that autotools know it should link c++ runtime with it
[Wednesday 01 September 2010] [10:04:11] <sustrik> (needed by 0mq library, in case of static linking)
[Wednesday 01 September 2010] [10:04:15] <pieterh> yes, indeed
[Wednesday 01 September 2010] [10:04:24] <pieterh> great
[Wednesday 01 September 2010] [10:04:42] <guido_g> why is the logging now a special transport?
[Wednesday 01 September 2010] [10:04:47] <pieterh> will have to use a naming convention to avoid conflicts, NP
[Wednesday 01 September 2010] [10:05:05] <sustrik> guido_g: two reasons
[Wednesday 01 September 2010] [10:05:39] <sustrik> 1. it’s a different beast, communicating with 0mq internals rather than communicating with a socket in a different thread
[Wednesday 01 September 2010] [10:05:50] <sustrik> 2. full backward compatibility
[Wednesday 01 September 2010] [10:06:14] <guido_g> ok
[Wednesday 01 September 2010] [10:06:49] <pieterh> sustrik: i see one problem with making it a separate transport
[Wednesday 01 September 2010] [10:06:54] <guido_g> how will users attach to this type of transport?
[Wednesday 01 September 2010] [10:07:08] <sustrik> guido_g: connect
[Wednesday 01 September 2010] [10:07:10] <guido_g> just a SUB w/ sys://log?
[Wednesday 01 September 2010] [10:07:16] <sustrik> pieterh: what problem?
[Wednesday 01 September 2010] [10:07:25] <sustrik> guido_g: yes
[Wednesday 01 September 2010] [10:07:31] <pieterh> take the syslog service as an analogy
[Wednesday 01 September 2010] [10:07:31] <guido_g> great! thanks
[Wednesday 01 September 2010] [10:07:40] <pieterh> it’s a bus, any component can participate
[Wednesday 01 September 2010] [10:08:08] <pieterh> ok, that’s not quite accurate but I mean you might want to allow arbitrary components to inject traffic into this
[Wednesday 01 September 2010] [10:08:19] <sustrik> i don’t :)
[Wednesday 01 September 2010] [10:08:24] <pieterh> and have it picked up by log clients without them needing to be changed
[Wednesday 01 September 2010] [10:08:25] <guido_g> :)
[Wednesday 01 September 2010] [10:08:46] <sustrik> what i mean is how would you scale that kind of thing?
[Wednesday 01 September 2010] [10:08:52] <sustrik> right now it’s easy
[Wednesday 01 September 2010] [10:09:06] <pieterh> it’s inproc or ipc, what’s the scaling issue?
[Wednesday 01 September 2010] [10:09:06] <sustrik> it’s constrained to the process
[Wednesday 01 September 2010] [10:09:17] <sustrik> the business logic
[Wednesday 01 September 2010] [10:09:24] <pieterh> attach device to sub, export as pub... np
[Wednesday 01 September 2010] [10:09:35] <sustrik> what you want with a business logic is a distributed logging service
[Wednesday 01 September 2010] [10:09:44] <guido_g> ack
[Wednesday 01 September 2010] [10:09:58] <guido_g> the way with a forwarder is ok
[Wednesday 01 September 2010] [10:10:07] <pieterh> i can play devil’s advocate here
[Wednesday 01 September 2010] [10:10:22] <sustrik> pieterh: i have no idea how you would implement it
[Wednesday 01 September 2010] [10:10:28] <pieterh> it looks like you are claiming a semantic difference in order to solve a naming clash
[Wednesday 01 September 2010] [10:10:40] <pieterh> it’s unclear what the value is of a new transport
[Wednesday 01 September 2010] [10:10:55] <pieterh> no clear problem you’re solving wrt to using inproc:
[Wednesday 01 September 2010] [10:10:55] <sustrik> you mean between sys & inproc or between 0mq logging and application logging?
[Wednesday 01 September 2010] [10:10:58] <pieterh> or ipc:
[Wednesday 01 September 2010] [10:12:22] <pieterh> sustrik: ... sorry, I don’t follow this thread
[Wednesday 01 September 2010] [10:12:36] <sustrik> pieterh> it looks like you are claiming a semantic difference in order to solve a naming clash
[Wednesday 01 September 2010] [10:12:45] <pieterh> ah, yes
[Wednesday 01 September 2010] [10:12:55] <pieterh> between sys: and inproc or ipc
[Wednesday 01 September 2010] [10:13:15] <pieterh> is sys: like inproc or like ipc?
[Wednesday 01 September 2010] [10:13:28] <sustrik> it’s different from both
[Wednesday 01 September 2010] [10:13:38] <sustrik> it has no peer actually
[Wednesday 01 September 2010] [10:13:55] <pieterh> yes, but can you connect to it from another process?
[Wednesday 01 September 2010] [10:13:59] <pieterh> is it an in or ex-process endpoint?
[Wednesday 01 September 2010] [10:14:12] <sustrik> it’s process-scoped
[Wednesday 01 September 2010] [10:14:19] <pieterh> so it’s like inproc
[Wednesday 01 September 2010] [10:14:30] <sustrik> sure, same scope
[Wednesday 01 September 2010] [10:14:33] <sustrik> different semantics
[Wednesday 01 September 2010] [10:15:01] <pieterh> now if i am being evil and minimalist i’ll ask what the value is in defining a totally new protocol instead of a protected namespace within inproc
[Wednesday 01 September 2010] [10:15:15] <pieterh> given that every new construct is something expensive to document, understand...
[Wednesday 01 September 2010] [10:15:45] <sustrik> same question: why not make ipc a separate namespace within tcp?
[Wednesday 01 September 2010] [10:15:51] <pieterh> but the semantics fit perfectly to an inproc SUB socket from the users’ POV
[Wednesday 01 September 2010] [10:16:09] <sustrik> no: there’s no peer socket
[Wednesday 01 September 2010] [10:16:23] <pieterh> that’s an implementation detail
[Wednesday 01 September 2010] [10:16:29] <pieterh> that’s behind the wall, and not relevant to users
[Wednesday 01 September 2010] [10:16:35] <pieterh> it looks like there is a peer socket
[Wednesday 01 September 2010] [10:16:38] <sustrik> ok, let me explain the internal mechanism of sys
[Wednesday 01 September 2010] [10:16:48] <pieterh> ah, hang on
[Wednesday 01 September 2010] [10:16:55] <pieterh> i’m at the API level
[Wednesday 01 September 2010] [10:17:04] <pieterh> internal mechanisms should not be relevant
[Wednesday 01 September 2010] [10:17:09] <guido_g> this is what sys: says imho, don’t expect the usual, this comming from the kernel
[Wednesday 01 September 2010] [10:17:24] <pieterh> but why?
[Wednesday 01 September 2010] [10:17:27] <pieterh> why the surprise?
[Wednesday 01 September 2010] [10:17:47] <guido_g> i can’t see any, sorry
[Wednesday 01 September 2010] [10:17:55] <pieterh> why is the kernel a ‘special’ component with its own transport that looks exactly like inproc: but has a different name?
[Wednesday 01 September 2010] [10:17:57] <pieterh> it’s surprising
[Wednesday 01 September 2010] [10:18:03] <sustrik> it’s administration/business logic distinction
[Wednesday 01 September 2010] [10:18:10] <sustrik> inproc carries business logic
[Wednesday 01 September 2010] [10:18:17] <sustrik> sys carries admin data
[Wednesday 01 September 2010] [10:18:18] <pieterh> not necessarily
[Wednesday 01 September 2010] [10:18:24] <sustrik> necessarily
[Wednesday 01 September 2010] [10:18:24] <pieterh> as soon as i make a framework over 0MQ
[Wednesday 01 September 2010] [10:18:30] <pieterh> then inproc carries admin data
[Wednesday 01 September 2010] [10:18:39] <guido_g> for an application using the core
[Wednesday 01 September 2010] [10:18:50] <guido_g> not the mq core itself
[Wednesday 01 September 2010] [10:18:58] <pieterh> ok... perhaps it would help if there was some significant semantic difference between admin data and business data
[Wednesday 01 September 2010] [10:19:01] <pieterh> such as priority
[Wednesday 01 September 2010] [10:19:02] <guido_g> i see sys: as the core namespace
[Wednesday 01 September 2010] [10:19:07] <pieterh> or blocking/ error handling
[Wednesday 01 September 2010] [10:19:19] <pieterh> guido_g: as a namespace this is a lousy solution IMO
[Wednesday 01 September 2010] [10:19:36] <pieterh> namespaces should be orthogonal to transports
[Wednesday 01 September 2010] [10:19:36] <guido_g> pieterh: why?
[Wednesday 01 September 2010] [10:19:57] <pieterh> well, this makes it impossible to run a separate logger to inspect a running process
[Wednesday 01 September 2010] [10:20:13] <pieterh> every single application needs a special thread that exports the sys: data over ipc: or tcp:
[Wednesday 01 September 2010] [10:20:17] <sustrik> the main problem i have with the whole log thing is that people are now going to use it for their business logic and thus break the patterns
[Wednesday 01 September 2010] [10:20:26] <sustrik> if so, i’ll rather remove it completely
[Wednesday 01 September 2010] [10:20:28] <pieterh> sustrik: no, you can prevent that easily
[Wednesday 01 September 2010] [10:20:36] <sustrik> ?
[Wednesday 01 September 2010] [10:20:36] <guido_g> sustrik: hold it, we need it!
[Wednesday 01 September 2010] [10:20:38] <pieterh> you can prevent that trivially
[Wednesday 01 September 2010] [10:20:43] <pieterh> use a protected name space
[Wednesday 01 September 2010] [10:20:56] <guido_g> like sys: ? :)
[Wednesday 01 September 2010] [10:21:00] <pieterh> recall in amqp how exchanges beginning with amq. were protected
[Wednesday 01 September 2010] [10:21:10] <pieterh> transports are not namespaces... :-(
[Wednesday 01 September 2010] [10:21:20] <guido_g> sure
[Wednesday 01 September 2010] [10:21:27] <sustrik> what i mean is people are now going to ask me to report disconnections via sys
[Wednesday 01 September 2010] [10:21:29] <pieterh> inproc://zmq/log and ipc://zmq/log would be perfect
[Wednesday 01 September 2010] [10:21:31] <pieterh> allows me to start separate logger
[Wednesday 01 September 2010] [10:21:38] <sustrik> so that they can handle each connection separately
[Wednesday 01 September 2010] [10:21:59] <pieterh> perhaps
[Wednesday 01 September 2010] [10:22:08] <sustrik> to prevent that, the admin part should be as much separated from the business part as possible
[Wednesday 01 September 2010] [10:22:09] <pieterh> that’s a use case but it’s possibly not the right solution
[Wednesday 01 September 2010] [10:22:18] <pieterh> hmm
[Wednesday 01 September 2010] [10:22:22] <sustrik> different API would be ideal
[Wednesday 01 September 2010] [10:22:36] <guido_g> but the application developer doe not know what makes sys: work technically, so the obvious similarity to inproc might be wrong
[Wednesday 01 September 2010] [10:22:36] <sustrik> but that’s too much work
[Wednesday 01 September 2010] [10:22:38] <pieterh> sustrik: do you see the value in allowing this namespace to work over both ipc and inproc?
[Wednesday 01 September 2010] [10:22:48] <sustrik> no
[Wednesday 01 September 2010] [10:23:07] <sustrik> it’s just adminstration stuff
[Wednesday 01 September 2010] [10:23:37] <pieterh> i don’t think logging should be mixed with out of band control of sockets
[Wednesday 01 September 2010] [10:23:45] <pieterh> that’s a different issue
[Wednesday 01 September 2010] [10:23:47] <sustrik> right
[Wednesday 01 September 2010] [10:23:55] <sustrik> but it’s an obvious trap
[Wednesday 01 September 2010] [10:24:11] <pieterh> avoidable trap
[Wednesday 01 September 2010] [10:24:13] <sustrik> i think someone have already suggested something like that
[Wednesday 01 September 2010] [10:24:19] <pieterh> yes
[Wednesday 01 September 2010] [10:24:27] <sustrik> i can avoid it by saying “NO”
[Wednesday 01 September 2010] [10:24:36] <sustrik> as i am core maintainer atm
[Wednesday 01 September 2010] [10:24:46] <pieterh> indeed
[Wednesday 01 September 2010] [10:24:53] <pieterh> but that’s not going away, your veto
[Wednesday 01 September 2010] [10:24:57] <sustrik> no mixing of business logic and administration
[Wednesday 01 September 2010] [10:25:04] <pieterh> anyone can fork 0MQ and add whatever they like
[Wednesday 01 September 2010] [10:25:10] <pieterh> but it won’t be the official version
[Wednesday 01 September 2010] [10:25:16] <sustrik> sure, that’s ok
[Wednesday 01 September 2010] [10:25:19] <pieterh> unless and until you vote it in
[Wednesday 01 September 2010] [10:25:27] <pieterh> no exceptions to this, period.
[Wednesday 01 September 2010] [10:25:58] <guido_g> are we still at the logging thing or do i have to move myself to a safe place?
[Wednesday 01 September 2010] [10:26:00] <sustrik> good, so unless i hear a good argument for mixing business logic and maitenence/debugging stuff, i would keep it apart
[Wednesday 01 September 2010] [10:26:18] <sustrik> it’s logging still :)
[Wednesday 01 September 2010] [10:26:27] <pieterh> sustrik: i’d like, really love, to be able to do logging from a separate process
[Wednesday 01 September 2010] [10:26:42] <pieterh> otherwise every application becomes N x more complex
[Wednesday 01 September 2010] [10:26:42] <sustrik> write an app that does that
[Wednesday 01 September 2010] [10:26:43] <sustrik> np
[Wednesday 01 September 2010] [10:26:50] <sustrik> 5 lines of code
[Wednesday 01 September 2010] [10:27:14] <pieterh> you said it was in process
[Wednesday 01 September 2010] [10:27:25] <pieterh> did I misunderstand you?
[Wednesday 01 September 2010] [10:27:35] <sustrik> you can create a device that would read from sys and republish on tcp
[Wednesday 01 September 2010] [10:27:44] <sustrik> you can even use forwarder
[Wednesday 01 September 2010] [10:28:19] <sustrik> when you do that you are effectively converting 0mq admin data into business logic data
[Wednesday 01 September 2010] [10:28:20] <sustrik> which is ok
[Wednesday 01 September 2010] [10:28:30] <sustrik> but at 0mq level i need them separated
[Wednesday 01 September 2010] [10:28:36] <sustrik> to avoid contamination
[Wednesday 01 September 2010] [10:28:50] <guido_g> if anyone can access the internal data flow w/o authorization it would be a nightmare
[Wednesday 01 September 2010] [10:29:05] <sustrik> you mean the logs?
[Wednesday 01 September 2010] [10:29:09] <guido_g> yes
[Wednesday 01 September 2010] [10:29:16] <sustrik> :(
[Wednesday 01 September 2010] [10:29:26] <sustrik> no authorisation as for now
[Wednesday 01 September 2010] [10:29:33] <sustrik> otoh
[Wednesday 01 September 2010] [10:29:38] <sustrik> it’s in-process only
[Wednesday 01 September 2010] [10:29:49] <sustrik> so nobody from outside can read from sys
[Wednesday 01 September 2010] [10:29:52] <guido_g> given the “one forwarder per program” model, the program decides on what to publish and where etc.
[Wednesday 01 September 2010] [10:30:31] <guido_g> sustrik: sure, but as far as i understood, pieterh wanted access from an external process w/o a forwarder inbetween
[Wednesday 01 September 2010] [10:30:47] <sustrik> yes, that was the original discussion
[Wednesday 01 September 2010] [10:31:10] <sustrik> actually it would be almost impossible from implementation point of view
[Wednesday 01 September 2010] [10:31:20] <sustrik> that’s black magic going on in the background :)
[Wednesday 01 September 2010] [10:31:21] <guido_g> details! :)
[Wednesday 01 September 2010] [10:31:44] <sustrik> well, sys is technically different from all the other sockets
[Wednesday 01 September 2010] [10:31:56] <sustrik> maybe pgm excluded
[Wednesday 01 September 2010] [10:32:02] <sustrik> becasue it’s a real bus
[Wednesday 01 September 2010] [10:32:08] <sustrik> ie. multiple writers
[Wednesday 01 September 2010] [10:32:13] <sustrik> multiple readers
[Wednesday 01 September 2010] [10:32:19] <sustrik> no device in the middle
[Wednesday 01 September 2010] [10:32:28] <guido_g> as i said “<guido_g> but the application developer doe not know what makes sys: work technically, so the obvious similarity to inproc might be wrong”
[Wednesday 01 September 2010] [10:32:56] <guido_g> therefore the sys: transport makes sense in my eyes
[Wednesday 01 September 2010] [10:33:09] <sustrik> yes, that was my argument as well
[Wednesday 01 September 2010] [10:33:14] <guido_g> good
[Wednesday 01 September 2010] [10:33:33] <guido_g> because i really like the logging
[Wednesday 01 September 2010] [10:33:48] <sustrik> it’s good but can be dangerous if misused
[Wednesday 01 September 2010] [10:34:04] <guido_g> mq is notoriously weak on giving information on what’s going on
[Wednesday 01 September 2010] [10:34:14] <sustrik> :|
[Wednesday 01 September 2010] [10:34:23] <guido_g> that’s a thing nearly every library has
[Wednesday 01 September 2010] [10:34:35] <sustrik> right
[Wednesday 01 September 2010] [10:34:52] <sustrik> that’s because it’s much more async that probably any other library around
[Wednesday 01 September 2010] [10:35:02] <sustrik> zillions of moving parts
[Wednesday 01 September 2010] [10:35:09] <sustrik> even without proper synchronisation
[Wednesday 01 September 2010] [10:35:12] <guido_g> and for real distributed applications there is rule 0: log everything, all the time
[Wednesday 01 September 2010] [10:35:21] <sustrik> right
[Wednesday 01 September 2010] [10:35:27] <sustrik> so, this should solve it
[Wednesday 01 September 2010] [10:35:32] <sustrik> hopefully
[Wednesday 01 September 2010] [10:35:47] <guido_g> it’s at least a clean interface
[Wednesday 01 September 2010] [10:36:01] <pieterh> sustrik: makes sense but it’s a shame that we’d need a dedicated thread in every app to export log data
[Wednesday 01 September 2010] [10:36:10] <pieterh> still, this can be extended over time
[Wednesday 01 September 2010] [10:36:23] <pieterh> minimalism demands I shut up now :-)
[Wednesday 01 September 2010] [10:36:41] <pieterh> though a new question comes to mind...
[Wednesday 01 September 2010] [10:36:56] <pieterh> if a bus transport is useful for 0MQ internally
[Wednesday 01 September 2010] [10:37:01] <pieterh> surely it is also useful for applications as well
[Wednesday 01 September 2010] [10:37:07] <pieterh> :-)
[Wednesday 01 September 2010] [10:37:09] <sustrik> it doesn’t scale
[Wednesday 01 September 2010] [10:37:17] <sustrik> it can work on level on 1 process
[Wednesday 01 September 2010] [10:37:24] <sustrik> even on LAN
[Wednesday 01 September 2010] [10:37:35] <sustrik> once you try to stretch it further it breaks
[Wednesday 01 September 2010] [10:38:00] <sustrik> the number of transfers grows quadratically with increasding number of components
[Wednesday 01 September 2010] [10:38:38] <sustrik> pieterh: ad dedicated thread
[Wednesday 01 September 2010] [10:38:42] <pieterh> sustrik: as an inproc bus, I meant
[Wednesday 01 September 2010] [10:38:45] <sustrik> yes, that’s why it’s a bus
[Wednesday 01 September 2010] [10:38:46] <pieterh> bus://
[Wednesday 01 September 2010] [10:38:58] <sustrik> to avoid speacial thread
[Wednesday 01 September 2010] [10:39:14] <sustrik> yes, can be
[Wednesday 01 September 2010] [10:39:37] <sustrik> but i like that sys is explicit abour “don’t pass your business logic through this”
[Wednesday 01 September 2010] [10:39:48] <pieterh> for some reason ‘magic system stuff you use here’ seems not ideal
[Wednesday 01 September 2010] [10:40:16] <pieterh> it’s like the core gets to use magic stuff the apps don’t get
[Wednesday 01 September 2010] [10:40:28] <sustrik> yes
[Wednesday 01 September 2010] [10:40:31] <guido_g> that’s why it’s the core
[Wednesday 01 September 2010] [10:40:37] <sustrik> because core doesn’t have to scale
[Wednesday 01 September 2010] [10:40:42] <sustrik> applications do
[Wednesday 01 September 2010] [10:41:36] <pieterh> imo you will get people demanding access to this transport for their essential internal signalling
[Wednesday 01 September 2010] [10:41:51] <pieterh> but i have no proof, just baseless speculation
[Wednesday 01 September 2010] [10:42:07] <sustrik> i expect it will be so
[Wednesday 01 September 2010] [10:42:19] <sustrik> thus i want it be named something really scary
[Wednesday 01 September 2010] [10:42:30] <guido_g> but the line is drawn “sorry, no access for you here”
[Wednesday 01 September 2010] [10:42:34] <pieterh> e.g. “how do I do a clean shutdown”... answer: signal every thread
[Wednesday 01 September 2010] [10:42:44] <guido_g> as with system calls into the kernel
[Wednesday 01 September 2010] [10:42:45] <pieterh> bah. that was the idea with PAIR as well
[Wednesday 01 September 2010] [10:42:49] <sustrik> that’s baked into 0mq as well
[Wednesday 01 September 2010] [10:42:52] <sustrik> ETERM
[Wednesday 01 September 2010] [10:42:55] <pieterh> but it turns out we really need PAIR for some use cases
[Wednesday 01 September 2010] [10:43:15] <sustrik> ok, the use case for pair:
[Wednesday 01 September 2010] [10:43:18] <pieterh> yeah, i know, it’s a bad example
[Wednesday 01 September 2010] [10:43:26] <sustrik> you have an application that does accounting and HR
[Wednesday 01 September 2010] [10:43:34] <sustrik> you want to split it into two applications
[Wednesday 01 September 2010] [10:43:37] <sustrik> one does accounting
[Wednesday 01 September 2010] [10:43:40] <sustrik> other does HR
[Wednesday 01 September 2010] [10:43:52] <sustrik> they have a PAIR socket between them
[Wednesday 01 September 2010] [10:44:10] <sustrik> because each of the peer is singleton
[Wednesday 01 September 2010] [10:44:16] <sustrik> and doesn’t scale
[Wednesday 01 September 2010] [10:44:23] <pieterh> my use case for PAIR is just interthread signalling
[Wednesday 01 September 2010] [10:44:32] <sustrik> hm
[Wednesday 01 September 2010] [10:44:37] <sustrik> what use case?
[Wednesday 01 September 2010] [10:44:52] <pieterh> i explained it in the guide, end of chapter 2
[Wednesday 01 September 2010] [10:45:00] <pieterh> thread a wants to signal to thread b that it’s done
[Wednesday 01 September 2010] [10:45:05] <pieterh> no scaling required
[Wednesday 01 September 2010] [10:45:10] <sustrik> let me see
[Wednesday 01 September 2010] [10:45:14] <pieterh> no reconnection required
[Wednesday 01 September 2010] [10:45:25] <pieterh> PAIR gives the best semantics afaics
[Wednesday 01 September 2010] [10:47:48] * sustrik is looking at the program
[Wednesday 01 September 2010] [10:48:00] <sustrik> this one: “Thread Coordination” ?
[Wednesday 01 September 2010] [10:51:38] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Wednesday 01 September 2010] [10:51:55] <sustrik> well, there’s no business logic in the example
[Wednesday 01 September 2010] [10:52:09] <sustrik> so it’s hard to say what’s the right socket types to use
[Wednesday 01 September 2010] [10:52:39] <sustrik> it the three components are by definition singletons, then PAIR is appropriate
[Wednesday 01 September 2010] [10:52:54] <sustrik> it say step2 can be parallelised the pipeline would be better
[Wednesday 01 September 2010] [10:53:10] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [10:53:49] <sustrik> the problem i have with pair is that it solves a problem (the accounting vs. HR problem)
[Wednesday 01 September 2010] [10:53:54] <sustrik> but it doesn’t scale
[Wednesday 01 September 2010] [10:54:29] <sustrik> by throwing more boxes on your accounting/HR application you won’t get more power
[Wednesday 01 September 2010] [10:54:43] <sustrik> even thought you paired the two by a PAIR socket
[Wednesday 01 September 2010] [11:17:37] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [11:17:48] Nick
pieterh_ is now known as pieterh.
[Wednesday 01 September 2010] [11:22:08] Quit gebi has left this server (Ping timeout: 276 seconds).
[Wednesday 01 September 2010] [11:29:51] <guido_g> sustrik: latest master shows poll problem again
[Wednesday 01 September 2010] [11:38:58] Quit mrm2m has left this server (Quit: Leaving.).
[Wednesday 01 September 2010] [12:11:55] <ModusPwnens> hello
[Wednesday 01 September 2010] [12:18:26] <ModusPwnens> i have a question about the zmq_msg_size function
[Wednesday 01 September 2010] [12:26:24] <cremes> ModusPwnens: go ahead and ask
[Wednesday 01 September 2010] [12:26:33] <cremes> if someone knows the answer, they’ll respond
[Wednesday 01 September 2010] [12:26:38] <ModusPwnens> ok
[Wednesday 01 September 2010] [12:26:56] <ModusPwnens> so as i have said before, I am using google protocl buffers to encode messages and send them
[Wednesday 01 September 2010] [12:27:29] <ModusPwnens> and on the clientsize, I do message.size(), where message is the size of the object encoded by google protocol buffers
[Wednesday 01 September 2010] [12:27:33] <ModusPwnens> however, on the server side
[Wednesday 01 September 2010] [12:27:41] <ModusPwnens> I use the zmq_msg_size function
[Wednesday 01 September 2010] [12:27:55] <ModusPwnens> but no matter how large the message actually is, zmq_msg_size always gives 36 bytes
[Wednesday 01 September 2010] [12:28:14] <ModusPwnens> I am wondering not only why that is, but how to get the actual size of the payload of the message
[Wednesday 01 September 2010] [12:28:21] <cremes> ok
[Wednesday 01 September 2010] [12:28:48] <cremes> it sounds to me like your client is not allocating the message correctly
[Wednesday 01 September 2010] [12:29:11] <cremes> do you have a code snippet showing how you are allocating the zmq_msg_t and assigning your protocol buffer to it?
[Wednesday 01 September 2010] [12:29:36] <ModusPwnens> rc = zmq_msg_init_data(&msg, cstr, message.size(), NULL, NULL);
[Wednesday 01 September 2010] [12:29:41] <ModusPwnens> something like that?>
[Wednesday 01 September 2010] [12:30:43] <cremes> hold on a sec...
[Wednesday 01 September 2010] [12:30:51] <ModusPwnens> k
[Wednesday 01 September 2010] [12:32:32] <cremes> the variable ‘cstr’ points to the data; are you sure its size is the same as message.size() ?
[Wednesday 01 September 2010] [12:32:47] <Steve-o> shouldn’t it be message.ByteSize() in Google protobufs?
[Wednesday 01 September 2010] [12:33:26] * cremes thinks that is a good question too since i am unfamiliar with that api
[Wednesday 01 September 2010] [12:34:08] <ModusPwnens> hmm, that’s a good point. I wasn’t aware that function existed..
[Wednesday 01 September 2010] [12:34:26] <ModusPwnens> but in any case, that doesn’t seem like it would solve my problem.
[Wednesday 01 September 2010] [12:35:03] <cremes> does the server always receive packets of length 36 exactly? if so, the client must be producing messages of that length
[Wednesday 01 September 2010] [12:35:25] <cremes> focus on the client side for now... something is likely wrong there
[Wednesday 01 September 2010] [12:35:59] <cremes> on the server, are the bytes received garbage? try printing the data the client sends and comparing it to the data the server receives
[Wednesday 01 September 2010] [12:36:10] <ModusPwnens> It seems so, but when i print the value of message.size() in the client, it prints outs omething much larger
[Wednesday 01 September 2010] [12:36:16] <ModusPwnens> and the bytes received are not garbage
[Wednesday 01 September 2010] [12:36:21] <ModusPwnens> i have verified the message coming through is valid
[Wednesday 01 September 2010] [12:37:17] <cremes> why not try: rc = zmq_msg_init_data(&msg, cstr, strlen(cstr), NULL, NULL);
[Wednesday 01 September 2010] [12:37:48] <cremes> i don’t understand the relationship between cstr and message.size()
[Wednesday 01 September 2010] [12:38:21] <ModusPwnens> Hmm, i changed that for a reason...
[Wednesday 01 September 2010] [12:38:32] <ModusPwnens> I think it’s because the zmq syntax doesn’t want a string there or something
[Wednesday 01 September 2010] [12:38:41] <ModusPwnens> so I had to convert a string into a c-style string
[Wednesday 01 September 2010] [12:38:46] <ModusPwnens> and thats what cstr is
[Wednesday 01 September 2010] [12:39:08] <cremes> ah ha... that is your problem
[Wednesday 01 September 2010] [12:39:19] <ModusPwnens> it is?
[Wednesday 01 September 2010] [12:39:20] <cremes> c style strings are terminated by a NUL 0
[Wednesday 01 September 2010] [12:39:50] <ModusPwnens> Yeah. Does that interfere with zeromq somehow?
[Wednesday 01 September 2010] [12:39:59] <cremes> hmmm, wait, i got ahead of myself there...
[Wednesday 01 September 2010] [12:40:14] <cremes> i’m going to think out loud
[Wednesday 01 September 2010] [12:40:17] <ModusPwnens> ok
[Wednesday 01 September 2010] [12:40:31] <cremes> so a c style string is really just a char* or a pointer to a byte buffer
[Wednesday 01 September 2010] [12:40:45] <ModusPwnens> correct
[Wednesday 01 September 2010] [12:40:48] <ModusPwnens> oh right
[Wednesday 01 September 2010] [12:40:51] <cremes> you need to pass a byte buffer pointing to the beginning of your encoded message
[Wednesday 01 September 2010] [12:40:52] <ModusPwnens> sorry you’re thinking out loud >_<
[Wednesday 01 September 2010] [12:41:06] <cremes> is the cstr pointing to that?
[Wednesday 01 September 2010] [12:41:13] <ModusPwnens> yeah
[Wednesday 01 September 2010] [12:41:28] <ModusPwnens> and then i strcpy the message into cstr
[Wednesday 01 September 2010] [12:41:29] <cremes> e.g. cstr = (char*) message.encode().get_pointer()
[Wednesday 01 September 2010] [12:41:30] <ModusPwnens> beforehand
[Wednesday 01 September 2010] [12:41:48] <ModusPwnens> yeah something like that
[Wednesday 01 September 2010] [12:41:49] <cremes> any embedded NULs will terminate the string copy
[Wednesday 01 September 2010] [12:42:06] <cremes> so you may only be copying 36 bytes even though your byte buffer is larger
[Wednesday 01 September 2010] [12:42:10] <cremes> use memcpy instead
[Wednesday 01 September 2010] [12:42:20] <cremes> does that sound reasonable?
[Wednesday 01 September 2010] [12:42:34] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [12:42:38] <ModusPwnens> but..wouldn’t that reflect on the serverside as far as displaying the message?
[Wednesday 01 September 2010] [12:42:47] <ModusPwnens> i.e. it wouldn’t display the whole thing?
[Wednesday 01 September 2010] [12:43:10] <ModusPwnens> furthermore
[Wednesday 01 September 2010] [12:43:18] <ModusPwnens> embedded nuls will terminate the string copy
[Wednesday 01 September 2010] [12:43:29] <ModusPwnens> but doesnt’ that only apply to the encoded message that is being copied?
[Wednesday 01 September 2010] [12:43:47] <cremes> ModusPwnens: you are going to have to show more code, i think
[Wednesday 01 September 2010] [12:44:15] <ModusPwnens> ok i will just copy the entire function into a pastebin
[Wednesday 01 September 2010] [12:44:19] <cremes> there are lots of folks successfully slinging around large strings and large byte buffers so there is likely an issue in your code
[Wednesday 01 September 2010] [12:44:27] <cremes> cool
[Wednesday 01 September 2010] [12:44:32] <ModusPwnens> That is definitely a possibility
[Wednesday 01 September 2010] [12:44:49] <cremes> you are using 2.0.8 release, yes? or the 2.1 master branch?
[Wednesday 01 September 2010] [12:44:58] <ModusPwnens> 2.0.7
[Wednesday 01 September 2010] [12:45:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r99ddfa7 10/ (builds/msvc/platform.hpp configure.in): maint: will become 2.0.9 -
http://bit.ly/dgicJb[Wednesday 01 September 2010] [12:45:26] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47aaf10 10/ :
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: * maint:
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: maint: will become 2.0.9
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: Conflicts:
[Wednesday 01 September 2010] [12:45:27] <CIA-20> zeromq2: builds/msvc/platform.hpp
[Wednesday 01 September 2010] [12:45:28] <CIA-20> zeromq2: 03Mikael Helbo Kjr 07master * r59315eb 10/ src/select.cpp :
[Wednesday 01 September 2010] [12:45:29] <CIA-20> zeromq2: Erasure of retired fd’s in select.cpp causes an assertion in MSVC 2008 STL
[Wednesday 01 September 2010] [12:45:29] <CIA-20> zeromq2: I was hitting an issue with an SCL enabled STL library in connection with the
[Wednesday 01 September 2010] [12:45:30] <CIA-20> zeromq2: way select_t::loop was erasing retired fd’s. The problem as identified by the
[Wednesday 01 September 2010] [12:45:31] <CIA-20> zeromq2: SCL assertion was that by the time the iterator given to the erase method was
[Wednesday 01 September 2010] [12:45:31] <CIA-20> zeromq2: called it was considered invalid by the library. I am not sure this isn’t just
[Wednesday 01 September 2010] [12:45:32] <CIA-20> zeromq2: a “quirk” of the MSVC STL library as the other code looks valid to me as well. -
http://bit.ly/dzUsD3[Wednesday 01 September 2010] [12:45:32] <CIA-20> zeromq2: 03Martin Sustrik 07master * ra81a373 10/ src/select.cpp :
[Wednesday 01 September 2010] [12:45:33] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 01 September 2010] [12:45:55] <cremes> 2.0.7 should be okay too; i don’t recall any data truncation bugs for any sockets
[Wednesday 01 September 2010] [12:46:35] <ModusPwnens> if i am doing anything stupid in general
[Wednesday 01 September 2010] [12:46:37] <ModusPwnens> please let me know
[Wednesday 01 September 2010] [12:46:59] <ModusPwnens> I am actually only a stupid so im not as experienced as most people here probably are..
[Wednesday 01 September 2010] [12:47:02] <ModusPwnens> wow
[Wednesday 01 September 2010] [12:47:07] <ModusPwnens> i meant student
[Wednesday 01 September 2010] [12:47:35] <cremes> line 36 confuses me...
[Wednesday 01 September 2010] [12:47:41] <ModusPwnens> oh yeah ignore that
[Wednesday 01 September 2010] [12:47:54] <cremes> is that supposed to be commented out?
[Wednesday 01 September 2010] [12:48:27] <cremes> because it is overwriting the size of the msg on the prior line
[Wednesday 01 September 2010] [12:48:37] <ModusPwnens> I think so. We put that in for debugging purposes.
[Wednesday 01 September 2010] [12:48:41] <cremes> with 36 bytes which matches up with how many you are receiving on the server side
[Wednesday 01 September 2010] [12:49:00] <cremes> well, that’s a bug
[Wednesday 01 September 2010] [12:49:06] <cremes> remove that and test again
[Wednesday 01 September 2010] [12:49:13] <ModusPwnens> sigh i hate asking stupid questions
[Wednesday 01 September 2010] [12:51:05] Quit plq has left this server (Remote host closed the connection).
[Wednesday 01 September 2010] [12:51:06] <cremes> don’t worry about it; you’ll be repaying the favor here in a few short weeks
[Wednesday 01 September 2010] [12:51:34] <ModusPwnens> ahhhhh gosh darn it
[Wednesday 01 September 2010] [12:51:35] <ModusPwnens> that was it
[Wednesday 01 September 2010] [12:51:41] <ModusPwnens> I’m sorry for wasting your time.
[Wednesday 01 September 2010] [12:51:45] Join jond has joined this channel (
~jon@87.74.16.24).
[Wednesday 01 September 2010] [12:51:45] <ModusPwnens> thanks for your help tho
[Wednesday 01 September 2010] [12:51:46] <cremes> btw, line 34 looks to me like a bug waiting to happen
[Wednesday 01 September 2010] [12:51:51] <cremes> i would use memcpy
[Wednesday 01 September 2010] [12:52:02] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [12:52:27] <ModusPwnens> because of the nul string thing?
[Wednesday 01 September 2010] [12:52:49] <cremes> right
[Wednesday 01 September 2010] [12:53:05] <cremes> memcpy(cstr, message.c_str(), message.size())
[Wednesday 01 September 2010] [12:53:11] <ModusPwnens> Ok, i will change it.
[Wednesday 01 September 2010] [12:53:13] <cremes> or use strncpy
[Wednesday 01 September 2010] [12:53:25] <cremes> but memcpy is likely quicker since it isn’t looking for NULs
[Wednesday 01 September 2010] [12:53:48] <cremes> ModusPwnens: and all this from a Ruby guy who hasn’t used C in anger for many years... :)
[Wednesday 01 September 2010] [12:54:10] <ModusPwnens> heh, well it looks like you still got it then.
[Wednesday 01 September 2010] [12:54:36] <ModusPwnens> oh, and i have another question this time
[Wednesday 01 September 2010] [12:54:48] <ModusPwnens> for the send and receive topology
[Wednesday 01 September 2010] [12:55:10] <ModusPwnens> is there a flag you can set or something that prevents the send and receive messages from blocking?
[Wednesday 01 September 2010] [12:55:54] <cremes> yes
[Wednesday 01 September 2010] [12:56:37] <cremes> btw, when using PUB sockets, it’s a good idea to put your topic in a separate message part
[Wednesday 01 September 2010] [12:57:00] <ModusPwnens> what do you mean?
[Wednesday 01 September 2010] [12:57:12] <cremes> the example code you pastied showed that you are sending the AddressBook as the first message part, so subscribers will match on it
[Wednesday 01 September 2010] [12:57:30] <cremes> i’ll pastie some pseudo-code showing what i mean
[Wednesday 01 September 2010] [12:57:39] <ModusPwnens> ok.
[Wednesday 01 September 2010] [12:58:46] <ModusPwnens> and as far as the no blocking, we want to use it so we can constantly send messages and then receive a message every once in a while, or vice-versa. But would it be better to just use multiple sockets and a publish-subscribe topology instead?
[Wednesday 01 September 2010] [12:58:47] <jond> cremes: if it’s std::string why not use the copy method std::string::copy?
[Wednesday 01 September 2010] [12:59:21] <mikko> do you actually need a copy there?
[Wednesday 01 September 2010] [12:59:58] <ModusPwnens> well i need to store the message I am sending in that string, so I would need to copy it in, right?
[Wednesday 01 September 2010] [13:01:33] <cremes> jond: i don’t know; i’m not a C++ guy but memcpy is pretty easy to figure out for me
[Wednesday 01 September 2010] [13:02:01] <cremes> mikko: you probably need the copy unless you can guarantee the buffer won’t go away until the message is actually sent
[Wednesday 01 September 2010] [13:02:46] <ModusPwnens> Hmm, that makes sense.
[Wednesday 01 September 2010] [13:02:52] <Steve-o> Does it really need to be a string? I used buf.SerializeToArray() and buf.ParseFromArray().
[Wednesday 01 September 2010] [13:03:56] <cremes> ModusPwnens: by sending a topic, your SUB sockets can use it to filter out unwanted messages
[Wednesday 01 September 2010] [13:04:27] <ModusPwnens> Yes that makes sense. I can see the usefulness of that.
[Wednesday 01 September 2010] [13:04:34] <ModusPwnens> And steve-o, you make a good point
[Wednesday 01 September 2010] [13:04:37] <jond> cremes: ok, there is a method on string, so you could do message.copy(cstr,message.size()), but the code id as you say mainly c
[Wednesday 01 September 2010] [13:04:54] <mikko> ModusPwnens: is that C++ or C ?
[Wednesday 01 September 2010] [13:05:06] <cremes> jond: i’m a ruby guy so take my C suggestions with a grain of salt ;)
[Wednesday 01 September 2010] [13:05:13] <ModusPwnens> well, I am using visual studio C++, but I am actually using the C bindings
[Wednesday 01 September 2010] [13:05:23] <mikko> cremes: yeah, true
[Wednesday 01 September 2010] [13:05:28] <mikko> on the earlier
[Wednesday 01 September 2010] [13:05:28] <ModusPwnens> mostly because I was having trouble with the C++ ones
[Wednesday 01 September 2010] [13:06:28] <ModusPwnens> and steve-o, that would probably be a better/more efficient way of doing this, right?
[Wednesday 01 September 2010] [13:06:43] <ModusPwnens> then I wouldn’t have to do all the c-string manipulation
[Wednesday 01 September 2010] [13:07:10] <Steve-o> it should be the best as of v2.1, but I think Google has changed their api a few times since
[Wednesday 01 September 2010] [13:09:06] <Steve-o> noting ByteSize() is only valid after you have actually filled in values for protobufs, it has caught me a few times
[Wednesday 01 September 2010] [13:09:24] <ModusPwnens> you mean after you have encoded a message?
[Wednesday 01 September 2010] [13:10:28] <Steve-o> I needed the message size before serializing
[Wednesday 01 September 2010] [13:11:28] <ModusPwnens> hmm
[Wednesday 01 September 2010] [13:11:32] <ModusPwnens> for the serializetoarray method
[Wednesday 01 September 2010] [13:11:49] Join gebi has joined this channel (
~gebi@84.119.80.149).
[Wednesday 01 September 2010] [13:12:10] <ModusPwnens> you would need to get the size of the data you are serializing before you can serialize it..?
[Wednesday 01 September 2010] [13:13:54] <Steve-o> For my code I need to reserve space in a socket buffer for the serialized form, so I need the size. Obviously usage case varies for everyone.
[Wednesday 01 September 2010] [13:14:43] <ModusPwnens> You mean the size is always known beforehand? What if you don’t know the size before, can you still use that?
[Wednesday 01 September 2010] [13:15:14] <Steve-o> probably if you know the protobuf object is always going to be smaller than your buffer
[Wednesday 01 September 2010] [13:15:45] <ModusPwnens> Hmm. okay.
[Wednesday 01 September 2010] [13:15:57] <Steve-o> it’s a bit safer to check the size before hand, much like vasprintf()
[Wednesday 01 September 2010] [13:19:44] <ModusPwnens> Yeah that makes sense.
[Wednesday 01 September 2010] [13:19:49] <ModusPwnens> Ok, thanks for your help guys!
[Wednesday 01 September 2010] [13:20:17] <ModusPwnens> I will probably be back again in a few days with something else..hopefully something less stupid.
[Wednesday 01 September 2010] [13:53:25] Quit xla has left this server (Quit: leaving).
[Wednesday 01 September 2010] [13:55:34] <pieterh> folks: if anyone wants to review
http://www.zeromq.com, feedback would be welcome
[Wednesday 01 September 2010] [13:55:45] <pieterh> this is meant to be the welcome site for new 0MQ users
[Wednesday 01 September 2010] [13:56:10] <pieterh> next I’ll remove a lot of the fluff from the zeromq.org front page and menus
[Wednesday 01 September 2010] [14:00:50] <pieterh> my idea is to make the zeromq.org site an index of all projects
[Wednesday 01 September 2010] [14:01:00] <pieterh> kind of ranked by activity somehow
[Wednesday 01 September 2010] [14:05:50] <pieterh> no-one minds? :-)
[Wednesday 01 September 2010] [14:09:03] Quit pieterh has left this server (Read error: Operation timed out).
[Wednesday 01 September 2010] [14:12:16] <Steve-o> inproc is still a new word
[Wednesday 01 September 2010] [14:13:32] <guido_g> the onhover bold is a little overdone
[Wednesday 01 September 2010] [14:15:46] Quit alfborge has left this server (Quit: leaving).
[Wednesday 01 September 2010] [14:16:00] <guido_g> on the page linked from 04READ should show the canonical doku first (the guide & the manual)
[Wednesday 01 September 2010] [14:16:56] <jond> steve-o: agree about inproc, not widely used AFAIK
[Wednesday 01 September 2010] [14:23:39] Quit pieterh has left this server (Ping timeout: 258 seconds).
[Wednesday 01 September 2010] [15:45:07] <mcxx> I got freshly cloned zeromq2 and pyzmq repos from github
[Wednesday 01 September 2010] [16:00:54] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 01 September 2010] [16:27:03] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 01 September 2010] [16:46:14] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Wednesday 01 September 2010] [17:27:20] Quit jond has left this server (Quit: Leaving).
[Wednesday 01 September 2010] [17:57:57] Quit jsimmons has left this server (Client Quit).
[Wednesday 01 September 2010] [18:27:24] Quit AndrewBC has left this server (Ping timeout: 260 seconds).
[Wednesday 01 September 2010] [18:38:18] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [18:42:55] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Wednesday 01 September 2010] [19:05:24] Quit plq has left this server (Ping timeout: 258 seconds).
[Wednesday 01 September 2010] [19:18:54] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 01 September 2010] [19:30:09] Quit plq has left this server (Ping timeout: 240 seconds).
[Wednesday 01 September 2010] [20:03:27] <ModusPwnens> Hi, I’m having an issue with the first byte of a message being corrupted when it is sent. Has anyone experienced this?
[Wednesday 01 September 2010] [20:05:43] <bgranger> mcxx: You need to grab the stable 2.0.7 releases of pyzmq and zeromq.
[Wednesday 01 September 2010] [20:06:02] <bgranger> Trunk of pyzmq works with stable 2.0.8 of zeromq.
[Wednesday 01 September 2010] [20:06:15] <bgranger> We have not started following the post 2.0.8 master of zeromq
[Wednesday 01 September 2010] [20:06:44] <ModusPwnens> clarification, it’s actually the first two bytes that get corrupted
[Wednesday 01 September 2010] [20:07:40] Quit MJW has left this server (Client Quit).
[Wednesday 01 September 2010] [20:08:42] Quit ModusPwnens has left this server.
[Wednesday 01 September 2010] [20:47:00] Quit
mw__ has left this server (Quit: Alea iactus est.).
[Wednesday 01 September 2010] [22:08:48] Quit miguelito has left this server (Read error: Connection reset by peer).
[Wednesday 01 September 2010] [22:40:34] <lestrrat> I assume this is the right place to talk about zeromq?
[Wednesday 01 September 2010] [22:42:31] <lestrrat> I just wanted to make if zeromq’s recv() is supposed to bail out properly (or not) when it receives a signal while it’s waiting for a message?
[Wednesday 01 September 2010] [22:42:49] <lestrrat> my perl binding seems to be stuck, and I’m trying to figure out if it’s my problem or not.
[Wednesday 01 September 2010] [23:46:46] <bgranger> lestrrat: I think I know what is going on.
[Wednesday 01 September 2010] [23:46:58] <bgranger> Are you still around?
[Wednesday 01 September 2010] [23:47:02] <lestrrat> yes!
[Wednesday 01 September 2010] [23:47:13] <bgranger> Hey, we have the same issue in the Python bindings.
[Wednesday 01 September 2010] [23:47:27] <bgranger> It perl is like Python, then it sets its own signal handlers.
[Wednesday 01 September 2010] [23:47:33] <lestrrat> ah, so glad I’m not alone on this :)
[Wednesday 01 September 2010] [23:47:34] <lestrrat> right
[Wednesday 01 September 2010] [23:47:47] <bgranger> BUT, while a blocking recv is waiting, perl is completely out of control.
[Wednesday 01 September 2010] [23:48:22] <AndrewBC> I wrapped mine in a try except in python, excepting a KeyboardInterrupt
[Wednesday 01 September 2010] [23:48:25] <AndrewBC> worked fine
[Wednesday 01 September 2010] [23:48:42] <bgranger> SO what happens is that Perl’s signal handler will receive the signal, but it won’t be handled until the recv returns. To test if this is what is really going on, try to create a simple example , then send the signal and send an additional msg to the socket so that recv returns.
[Wednesday 01 September 2010] [23:48:44] <AndrewBC> so you might have to manually catch ^C if that’s possible in perl
[Wednesday 01 September 2010] [23:49:14] <bgranger> AndrewBC, it won’t work if there is not a msg recv’d
[Wednesday 01 September 2010] [23:49:28] <bgranger> Or I should say, the KeyboardInt won’t be raised until recv returns.
[Wednesday 01 September 2010] [23:49:37] <lestrrat> bgranger: ah right
[Wednesday 01 September 2010] [23:49:37] <AndrewBC> oh, hm
[Wednesday 01 September 2010] [23:49:38] <bgranger> You can’t really interupt it with SIGINT
[Wednesday 01 September 2010] [23:50:22] <lestrrat> I can go and confirm, but if that’s the case, it’s really zeromq’s responsibility to return on a signal, no? (with errno = EINTER or whatever)
[Wednesday 01 September 2010] [23:50:35] <bgranger> But the same is true when Python calls any external C/C++ code.
[Wednesday 01 September 2010] [23:50:40] <lestrrat> right
[Wednesday 01 September 2010] [23:50:55] <bgranger> No, because Python’s signal handler is still in place, it just doesn’t get to run.
[Wednesday 01 September 2010] [23:51:20] <lestrrat> yeah, but we can propagate the signal as long as we get back control
[Wednesday 01 September 2010] [23:51:58] <bgranger> But you can’t get back control
[Wednesday 01 September 2010] [23:52:02] <lestrrat> hmm?
[Wednesday 01 September 2010] [23:52:26] <bgranger> The signal handler is the one at the C level, not the one that you can control using the signal module
[Wednesday 01 September 2010] [23:52:41] <lestrrat> yeah, but I’m writing the Perl<->C binding, so I have control :)
[Wednesday 01 September 2010] [23:53:01] <bgranger> To change the behavior you actually to have use signal.h and set jump/long jump logic.
[Wednesday 01 September 2010] [23:53:07] <bgranger> But it can be done
[Wednesday 01 September 2010] [23:54:00] <bgranger> It is just subtle logic and hard to get right in a cross platform manner.
[Wednesday 01 September 2010] [23:54:15] <bgranger> I don’t think any of the language bindings are doing this yet, but eventually all should
[Wednesday 01 September 2010] [23:54:46] <lestrrat> I dunno the python internals, but I don’t think we need a jump logic for perl
[Wednesday 01 September 2010] [23:54:49] <bgranger> Meanwhile, we are just not using blocking recv when it needs to be interupted
[Wednesday 01 September 2010] [23:55:32] <bgranger> If perl sets its own signal handlers at the C level, you likely will.
[Wednesday 01 September 2010] [23:55:35] <lestrrat> perl’s signal handling doesn’t run on “real time”. it’s deferred until the next perl op. so as long as I’m in C land, I can tell the Perl interpreter that there was a signal, and properly emulate it.
[Wednesday 01 September 2010] [23:56:33] <bgranger> That is how Python works as well and it why the default signal handler doesn’t actually interrupt “in real time:”
[Wednesday 01 September 2010] [23:56:58] <lestrrat> then I don’t see why you need a jump...? what am I missing?
[Wednesday 01 September 2010] [23:57:46] <bgranger> I don’t remember the details honestly. I think the issue is that you have to be very careful about restoring Python’s regular signal handler and putting the intepreter back in the right place in the stack.
[Wednesday 01 September 2010] [23:57:57] <lestrrat> hmm.
[Wednesday 01 September 2010] [23:58:25] <bgranger> That is, if you want control to return to Perl/Python in a way that doesn’t simply kill the process.
[Wednesday 01 September 2010] [23:58:38] <lestrrat> actually, I don’t really care either way – I don’t have to have zmq_recv to get back to the Perl handler – I just need it to get back to me once a signal is sent w/o killing the process
[Wednesday 01 September 2010] [23:58:45] <lestrrat> otherwise servers hang.
[Wednesday 01 September 2010] [23:59:01] <bgranger> yep
[Wednesday 01 September 2010] [23:59:14] <lestrrat> glad to know I’m the only one. I can safely write an RFE :)
[Thursday 02 September 2010] [00:43:40] <sustrik> bgranger, lestrrat: i am still not sure about EINTR behaviour
[Thursday 02 September 2010] [00:43:53] <sustrik> would returning EINTR from blocking call help in any way?
[Thursday 02 September 2010] [00:47:53] <bgranger> In what situations?
[Thursday 02 September 2010] [00:49:02] <bgranger> Are you thinking of installing a signal handler that catches SIGINT and translates that into EINTR?
[Thursday 02 September 2010] [00:49:33] <bgranger> That would help us, but that signal handler would have to play nice with that of Python
[Thursday 02 September 2010] [00:52:11] <sustrik> bgragner: no, i meant the peculiar functionality of 0mq
[Thursday 02 September 2010] [00:52:25] <sustrik> that catches OS’s EINTR
[Thursday 02 September 2010] [00:52:34] <sustrik> and instead of returning it to the user
[Thursday 02 September 2010] [00:52:45] <sustrik> just restarts the blocking operation
[Thursday 02 September 2010] [00:53:01] <bgranger> I am not familiar with what zmq is doing right now
[Thursday 02 September 2010] [00:53:07] <sustrik> but afaiu that’s not the problem you are dealing with
[Thursday 02 September 2010] [00:53:30] <bgranger> When does the OS signal EINTR?
[Thursday 02 September 2010] [00:53:47] <bgranger> And how does zeromq currently handle it?
[Thursday 02 September 2010] [00:54:00] <sustrik> if a threads gets a signal while in blocking operation
[Thursday 02 September 2010] [00:54:33] <sustrik> 0mq currently ignores it, i.e. restarts the blocking op
[Thursday 02 September 2010] [00:56:29] <bgranger> I guess what I don’t know if how all of this interplays with Python’s signal handling.
[Thursday 02 September 2010] [00:56:35] <sustrik> exactly
[Thursday 02 September 2010] [00:56:43] <bgranger> Python only receives signals in the main thread
[Thursday 02 September 2010] [00:56:55] <bgranger> Does zeromq install any signal handlers?
[Thursday 02 September 2010] [00:56:57] <sustrik> that’s the one that’s blocked in 0mq?
[Thursday 02 September 2010] [00:57:05] <sustrik> no, it does not
[Thursday 02 September 2010] [00:57:10] <bgranger> OK
[Thursday 02 September 2010] [00:57:21] <bgranger> but when a socket call returns EINTR, it just continues
[Thursday 02 September 2010] [00:57:28] <sustrik> right
[Thursday 02 September 2010] [00:57:34] <sustrik> actaully, you can check what’s happening yourself
[Thursday 02 September 2010] [00:57:37] <bgranger> OK, then another ?
[Thursday 02 September 2010] [00:57:41] <bgranger> OK
[Thursday 02 September 2010] [00:57:46] <bgranger> How...
[Thursday 02 September 2010] [00:57:46] <sustrik> when waiting for a message
[Thursday 02 September 2010] [00:57:59] <sustrik> 0mq is stuck in a read call
[Thursday 02 September 2010] [00:58:06] <bgranger> Correct
[Thursday 02 September 2010] [00:58:09] <sustrik> let me find the exact code...
[Thursday 02 September 2010] [00:58:18] <bgranger> OK great, that would be helpful.
[Thursday 02 September 2010] [00:59:27] <bgranger> Are the other occasions where EINTR is returned?
[Thursday 02 September 2010] [01:00:06] <sustrik> polling
[Thursday 02 September 2010] [01:00:12] <sustrik> are you on linux?
[Thursday 02 September 2010] [01:00:16] <lestrrat> sorry was away
[Thursday 02 September 2010] [01:00:21] <bgranger> linux and Mac
[Thursday 02 September 2010] [01:00:22] <sustrik> hi
[Thursday 02 September 2010] [01:00:31] <bgranger> Talking about signal handling more...
[Thursday 02 September 2010] [01:00:37] <sustrik> trunk or 2.0.8?
[Thursday 02 September 2010] [01:00:55] <bgranger> pyzmq doesn’t work with trunk yet, sso 2.0.8 or 2.0.7
[Thursday 02 September 2010] [01:00:58] <lestrrat> I’m using 2.0.8
[Thursday 02 September 2010] [01:01:53] <bgranger> but if we can do signal -> EINTR -> Python bindings -> raise KeyboardInterrupt that would be great
[Thursday 02 September 2010] [01:02:04] <sustrik> yes
[Thursday 02 September 2010] [01:02:14] <sustrik> you see the loop, right?
[Thursday 02 September 2010] [01:02:21] <lestrrat> right
[Thursday 02 September 2010] [01:02:28] <bgranger> Yep
[Thursday 02 September 2010] [01:02:33] <sustrik> just put a printf there or something
[Thursday 02 September 2010] [01:02:42] <sustrik> and check whether it loops in case of SIGINT
[Thursday 02 September 2010] [01:02:48] <bgranger> OK I will do that
[Thursday 02 September 2010] [01:02:56] <bgranger> I will check on Linux and OS X
[Thursday 02 September 2010] [01:02:57] <sustrik> if so we can return EINTR instead of looping
[Thursday 02 September 2010] [01:03:00] <sustrik> great
[Thursday 02 September 2010] [01:03:01] <sustrik> thanks
[Thursday 02 September 2010] [01:03:04] <bgranger> Cool
[Thursday 02 September 2010] [01:03:07] <bgranger> Thanks!
[Thursday 02 September 2010] [01:03:15] <bgranger> Won’t get to it tonight though
[Thursday 02 September 2010] [01:03:21] <bgranger> Getting late here...
[Thursday 02 September 2010] [01:03:39] <sustrik> sure
[Thursday 02 September 2010] [01:03:53] <sustrik> lestrrat: you may try the same, if in hurry
[Thursday 02 September 2010] [01:03:55] <lestrrat> yeah, I’m on $day_job too, so I’m not going to be able to check it right away. Will see if I can sneak that in in the next couple of hours
[Thursday 02 September 2010] [01:04:01] <bgranger> If I want to try the change, I can just have the function return EINTR?
[Thursday 02 September 2010] [01:04:11] <lestrrat> yep, will do when pointy haired boss isn’t looking
[Thursday 02 September 2010] [01:04:17] <sustrik> :)
[Thursday 02 September 2010] [01:04:18] <lestrrat> AFK for now
[Thursday 02 September 2010] [01:04:28] <bgranger> Will the return value of that function be returned to the caller
[Thursday 02 September 2010] [01:04:33] <sustrik> bgranger: probably
[Thursday 02 September 2010] [01:04:46] <sustrik> let me check
[Thursday 02 September 2010] [01:04:52] <bgranger> Well, it looks like it must return a bool though
[Thursday 02 September 2010] [01:05:07] <bgranger> May have to change more logic.
[Thursday 02 September 2010] [01:05:16] <sustrik> ah, ok
[Thursday 02 September 2010] [01:05:24] <sustrik> it’s not propagated
[Thursday 02 September 2010] [01:05:30] <sustrik> but i can fix that
[Thursday 02 September 2010] [01:05:36] <sustrik> the question is whether it would help
[Thursday 02 September 2010] [01:05:38] <bgranger> OK, let’s test first
[Thursday 02 September 2010] [01:05:40] <bgranger> right
[Thursday 02 September 2010] [01:09:00] <sustrik> lestrrat: are you in japan? i’ve notices some of your twitter notes
[Thursday 02 September 2010] [01:09:12] <sustrik> had no idea what they were about though :)
[Thursday 02 September 2010] [01:10:29] <guido_g> hi all
[Thursday 02 September 2010] [01:10:37] <guido_g> good morning sustrik
[Thursday 02 September 2010] [01:10:44] <sustrik> morning
[Thursday 02 September 2010] [01:11:11] <guido_g> did you read that the poll segfault in master is still there?
[Thursday 02 September 2010] [01:11:38] <sustrik> yep, seen you saying it yesterday
[Thursday 02 September 2010] [01:11:42] <sustrik> can i reproduce it?
[Thursday 02 September 2010] [01:11:50] <guido_g> i hope so
[Thursday 02 September 2010] [01:11:58] <sustrik> which test program?
[Thursday 02 September 2010] [01:12:28] <guido_g> zmqcpp and sender
[Thursday 02 September 2010] [01:12:42] <guido_g> zmqcpp segfaults in poll
[Thursday 02 September 2010] [01:13:01] <sustrik> what order to start them in?
[Thursday 02 September 2010] [01:13:20] <guido_g> i started zmqcpp first and then sender
[Thursday 02 September 2010] [01:13:26] <guido_g> sender is a one shot thing
[Thursday 02 September 2010] [01:13:30] <sustrik> ok
[Thursday 02 September 2010] [01:14:18] <guido_g> going to write a mq test program for that
[Thursday 02 September 2010] [01:14:24] <sustrik> ack
[Thursday 02 September 2010] [01:14:44] <guido_g> seems to be a nasty corner that needs some regression
[Thursday 02 September 2010] [01:22:52] <sustrik> guido_g:
[Thursday 02 September 2010] [01:22:54] <sustrik> In file included from zmqcpp.cpp:9:
[Thursday 02 September 2010] [01:22:54] <sustrik> worker.hpp:14:21: error: pqxx/pqxx: No such file or directory
[Thursday 02 September 2010] [01:22:56] <sustrik> ...
[Thursday 02 September 2010] [01:23:05] <guido_g> oh shit
[Thursday 02 September 2010] [01:23:07] <guido_g> sorry
[Thursday 02 September 2010] [01:23:14] <sustrik> I’ll rather wait for your test program
[Thursday 02 September 2010] [01:23:21] <guido_g> ack
[Thursday 02 September 2010] [01:23:29] <guido_g> never mind then
[Thursday 02 September 2010] [01:23:45] <guido_g> “i’ll be back!” :)
[Thursday 02 September 2010] [01:24:07] <sustrik> ;)
[Thursday 02 September 2010] [01:24:45] <guido_g> so, day job is waiting, cu
[Thursday 02 September 2010] [01:24:57] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [01:31:01] <lestrrat> sustrik: yes I’m in Japan :)
[Thursday 02 September 2010] [01:32:01] <sustrik> lot of japanese tweets about 0mq lately
[Thursday 02 September 2010] [01:32:04] <sustrik> it’s fun
[Thursday 02 September 2010] [01:33:44] <lestrrat> let’s see if I can work on that signal thing...
[Thursday 02 September 2010] [01:34:11] <sustrik> ok
[Thursday 02 September 2010] [01:40:09] <lestrrat> alright, so I’m a autoconf noob. how do I make this go away?:
[Thursday 02 September 2010] [01:40:16] Notice -lestrrat to #zeromq- ./configure: line 18331: syntax error near unexpected token
`GLIB,’
[Thursday 02 September 2010] [01:40:17] Notice -lestrrat to #zeromq- ./configure: line 18331: ` PKG_CHECK_MODULES(GLIB, glib-2.0 gthread-2.0)’
[Thursday 02 September 2010] [01:41:27] <sustrik> you need newer version of autoconf
[Thursday 02 September 2010] [01:41:34] <sustrik> are you on OSX?
[Thursday 02 September 2010] [01:42:40] <sustrik> this one would deserve going into FAQ btw
[Thursday 02 September 2010] [01:43:01] <lestrrat> yeah, I’m on OSX
[Thursday 02 September 2010] [01:43:18] <sustrik> it’s a known issue
[Thursday 02 September 2010] [01:43:22] <lestrrat> grrr
[Thursday 02 September 2010] [01:43:30] * lestrrat looks for autoconf in homebrew
[Thursday 02 September 2010] [01:43:34] <sustrik> osx has old version of autoconf installed by default
[Thursday 02 September 2010] [01:44:26] <lestrrat> grr, and it’s not there in homebrew
[Thursday 02 September 2010] [01:47:35] <lestrrat> man, this is going to take some time :)
[Thursday 02 September 2010] [01:47:37] <sustrik> lestrrat: sorry, it’s pkg-config
[Thursday 02 September 2010] [01:47:55] * sustrik is autotools noob as well
[Thursday 02 September 2010] [01:47:58] <lestrrat> oh cool
[Thursday 02 September 2010] [01:48:52] <lestrrat> hmm, I seem to have the latest pkg-config
[Thursday 02 September 2010] [01:49:25] <sustrik> are you building witg –with-pgm btw?
[Thursday 02 September 2010] [01:50:15] <lestrrat> nay, just the default with a custom –prefix
[Thursday 02 September 2010] [01:50:19] <sustrik> hm
[Thursday 02 September 2010] [01:50:28] <sustrik> well, you’ll have to ask on the mailing list
[Thursday 02 September 2010] [01:50:38] <sustrik> as i say, i’m an autotools idiot
[Thursday 02 September 2010] [01:53:06] <lestrrat> trying out autoconf anyways
[Thursday 02 September 2010] [02:09:07] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rd5b6f68 10/ AUTHORS : Mikael Kjaer added to AUTHORS -
http://bit.ly/aI6jV2[Thursday 02 September 2010] [02:09:08] <CIA-20> zeromq2: 03Bernd Melchers 07maint * r8ec0743 10/ (AUTHORS src/signaler.cpp): Fix for signaler_t on HP-UX and AIX platforms -
http://bit.ly/9XM0R0[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: 03Jon Dyte 07maint * r14853c2 10/ (src/prefix_tree.cpp src/prefix_tree.hpp):
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: This is because as it adds the 255th element at a node it attempts to calculate
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: the count member var which is an unsigned char via count = (255 -0) + 1; and
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: pass the result to realloc. Unfortunately the result is zero and realloc returns
[Thursday 02 September 2010] [02:09:12] <CIA-20> zeromq2: null; the prefix_tree asserts. I have fixed it by making the count an unsigned
[Thursday 02 September 2010] [02:09:35] <CIA-20> zeromq2: 03Martin Sustrik 07master * rd5b6f68 10/ AUTHORS : Mikael Kjaer added to AUTHORS -
http://bit.ly/aI6jV2[Thursday 02 September 2010] [02:09:36] <CIA-20> zeromq2: 03Bernd Melchers 07master * r8ec0743 10/ (AUTHORS src/signaler.cpp): Fix for signaler_t on HP-UX and AIX platforms -
http://bit.ly/9XM0R0[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: 03Jon Dyte 07master * r14853c2 10/ (src/prefix_tree.cpp src/prefix_tree.hpp):
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: This is because as it adds the 255th element at a node it attempts to calculate
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: the count member var which is an unsigned char via count = (255 -0) + 1; and
[Thursday 02 September 2010] [02:09:39] <CIA-20> zeromq2: pass the result to realloc. Unfortunately the result is zero and realloc returns
[Thursday 02 September 2010] [02:09:40] <CIA-20> zeromq2: null; the prefix_tree asserts. I have fixed it by making the count an unsigned
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0a1f7e3 10/ (AUTHORS src/signaler.cpp src/trie.cpp src/trie.hpp): (log message trimmed)
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Merge branch ‘maint’
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: * maint:
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Prior to this patch prefix_tree asserts.
[Thursday 02 September 2010] [02:10:03] <CIA-20> zeromq2: Fix for signaler_t on HP-UX and AIX platforms
[Thursday 02 September 2010] [02:10:04] <CIA-20> zeromq2: Mikael Kjaer added to AUTHORS
[Thursday 02 September 2010] [02:10:04] <CIA-20> zeromq2: Conflicts:
[Thursday 02 September 2010] [02:16:49] Quit bgranger has left this server (Quit: bgranger).
[Thursday 02 September 2010] [02:35:56] <lestrrat> hmm, zeromq HEAD against my perl binding doesn’t work – (seeing some tests block on odd places)
[Thursday 02 September 2010] [02:36:02] <lestrrat> could be something on my part, of course.
[Thursday 02 September 2010] [02:36:05] <lestrrat> investigating...
[Thursday 02 September 2010] [02:52:46] <lestrrat> hmm, switching back to 2.0.8 makes the problem go away. I see all my destructors running, so I’m willing to be there’s some sort of background thread or something that’s kept alive in zeromq
[Thursday 02 September 2010] [02:54:03] <guido_g> re
[Thursday 02 September 2010] [02:58:55] <sustrik> lestrrat: HEAD is a dev version, testing with 2.0.8 is ok
[Thursday 02 September 2010] [02:59:03] <lestrrat> k
[Thursday 02 September 2010] [02:59:12] <sustrik> what about the SIGINT thing?
[Thursday 02 September 2010] [02:59:49] <lestrrat> trying now
[Thursday 02 September 2010] [03:02:10] <lestrrat> so yeah, looks like it’s stuck in that loop
[Thursday 02 September 2010] [03:02:29] <lestrrat> tweaking to see if I can cleanly exit there...
[Thursday 02 September 2010] [03:05:37] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [03:11:16] <sustrik> lestrrat: i’m going to be away now for a while, but leave me a message here about whether returning EINTR from 0mq would help you
[Thursday 02 September 2010] [03:11:37] <lestrrat> yep, thanks
[Thursday 02 September 2010] [03:23:34] <lestrrat> this works for me!
[Thursday 02 September 2010] [03:41:43] Quit grahamalot has left this server (Quit: grahamalot).
[Thursday 02 September 2010] [04:30:52] Quit pieterh has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [04:45:31] <sustrik> lestrrat: great
[Thursday 02 September 2010] [04:45:51] <sustrik> i’ll fix it in a bit different way (actually returning EINTR)
[Thursday 02 September 2010] [04:46:04] <sustrik> but it’s good news to know it solves the Ctrl+C problem
[Thursday 02 September 2010] [04:46:40] <sustrik> we’ll see whether it helps with python as well
[Thursday 02 September 2010] [04:52:42] Quit gebi has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [04:58:39] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 02 September 2010] [05:00:31] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 02 September 2010] [05:19:30] Quit AndrewBC has left this server (Ping timeout: 258 seconds).
[Thursday 02 September 2010] [05:21:08] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [05:23:35] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [05:23:42] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 02 September 2010] [05:52:32] <mato> sustrik: are you there?
[Thursday 02 September 2010] [06:00:55] <sssss> test
[Thursday 02 September 2010] [06:01:15] Quit sssss has left this server (Client Quit).
[Thursday 02 September 2010] [06:06:57] <sustrik> mato: re
[Thursday 02 September 2010] [06:14:51] <mato> sustrik: could you switch on all the test servers + switch please?
[Thursday 02 September 2010] [06:15:15] <mato> sustrik: actually, just one of the 8-core boxes is all i need right now
[Thursday 02 September 2010] [06:15:32] <mato> sustrik: and the two white ones
[Thursday 02 September 2010] [06:15:51] <mato> sustrik: so pick the quieter of janos or csaba :) i don’t care...
[Thursday 02 September 2010] [06:16:23] <sustrik> oh my
[Thursday 02 September 2010] [06:16:24] <sustrik> ok
[Thursday 02 September 2010] [06:16:41] * sustrik is inserting earplugs
[Thursday 02 September 2010] [06:16:46] <mato> :)
[Thursday 02 September 2010] [06:16:59] <mato> loud music also works :)
[Thursday 02 September 2010] [06:18:13] <sustrik> started
[Thursday 02 September 2010] [06:18:30] <mato> damn
[Thursday 02 September 2010] [06:18:31] <sustrik> now i realise there’s just one of those big boxes here
[Thursday 02 September 2010] [06:18:38] <sustrik> where’s the other one?
[Thursday 02 September 2010] [06:18:39] <pieterh> hey, the lights just dimmed here...!
[Thursday 02 September 2010] [06:18:50] <sustrik> still at malosek’s place?
[Thursday 02 September 2010] [06:18:52] <mato> sustrik: just one? i dunno? does malo still have the other?
[Thursday 02 September 2010] [06:19:02] <mato> sustrik: i can’t connect to your IP at all :(
[Thursday 02 September 2010] [06:19:22] <mato> hmm
[Thursday 02 September 2010] [06:19:29] <sustrik> i see the world outside
[Thursday 02 September 2010] [06:19:55] <mato> i have a suspicion your chello connection is funny
[Thursday 02 September 2010] [06:20:08] <sustrik> ?
[Thursday 02 September 2010] [06:20:22] <mato> well, first of all, your ip is on some weird subnet
[Thursday 02 September 2010] [06:20:42] <mato> which i’ve never seen
[Thursday 02 September 2010] [06:20:49] <mato> 2nd, i can’t ping or connect to it :(
[Thursday 02 September 2010] [06:21:11] <sustrik> :(
[Thursday 02 September 2010] [06:21:31] <sustrik> you can take the boxes
[Thursday 02 September 2010] [06:34:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [06:44:06] <guido_g> openvpn is really nice for this kind of things
[Thursday 02 September 2010] [06:45:44] <sustrik> it works now (not sure why)
[Thursday 02 September 2010] [07:10:56] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 02 September 2010] [07:24:02] Quit pieterh has left this server (Ping timeout: 272 seconds).
[Thursday 02 September 2010] [07:26:13] <mato> wow
[Thursday 02 September 2010] [07:26:19] <mato> sustrik: quick question...
[Thursday 02 September 2010] [07:26:40] <mato> sustrik: zmq_free_fn() is called in the context of the application or i/o thread?
[Thursday 02 September 2010] [07:27:24] <sustrik> i/o
[Thursday 02 September 2010] [07:27:28] <mato> riiiiiight
[Thursday 02 September 2010] [07:27:33] <mato> that explains a lot
[Thursday 02 September 2010] [07:27:48] <sustrik> ?
[Thursday 02 September 2010] [07:28:05] <mato> i’m having issues with the python GIL
[Thursday 02 September 2010] [07:28:18] <mato> at least i think that’s what happening
[Thursday 02 September 2010] [07:28:24] <sustrik> a-ha, i see
[Thursday 02 September 2010] [07:28:31] <mato> it explains all the mysterious hanging close() with 2.0.x
[Thursday 02 September 2010] [07:28:38] <sustrik> python zero-copy functionality
[Thursday 02 September 2010] [07:28:43] <mato> ja
[Thursday 02 September 2010] [07:28:59] <mato> anyway, i’m going to lunch, we can discuss when i get back...
[Thursday 02 September 2010] [07:29:03] <sustrik> ok
[Thursday 02 September 2010] [07:38:29] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Thursday 02 September 2010] [07:47:22] <sustrik> thanks
[Thursday 02 September 2010] [07:47:37] <sustrik> have you thought of a pub to meet the other guys in?
[Thursday 02 September 2010] [07:47:50] <sustrik> it’s only 3 people so far
[Thursday 02 September 2010] [07:47:58] <sustrik> so no need for complex arrangements
[Thursday 02 September 2010] [07:48:18] <sustrik> say it’ll be 5-7 ultimately
[Thursday 02 September 2010] [07:48:49] <sustrik> oops :)
[Thursday 02 September 2010] [07:48:57] <sustrik> that was intended for another window
[Thursday 02 September 2010] [07:49:03] <sustrik> the london meetup stuff
[Thursday 02 September 2010] [08:16:57] <mikko> sustrik: theodore bullfrog is a decent place
[Thursday 02 September 2010] [08:18:58] <sustrik> looks central enough
[Thursday 02 September 2010] [08:20:46] <sustrik> is that restaurant or a pub?
[Thursday 02 September 2010] [08:24:12] <sustrik> mikko: hullo!
[Thursday 02 September 2010] [08:24:53] <mikko> pub
[Thursday 02 September 2010] [08:24:57] <mikko> but they serve food as well
[Thursday 02 September 2010] [08:25:27] <sustrik> ok, why not then
[Thursday 02 September 2010] [08:25:47] Quit guido_g has left this server (Disconnected by services).
[Thursday 02 September 2010] [08:25:59] Nick OldNugget is now known as guido_g.
[Thursday 02 September 2010] [08:26:31] <sustrik> nice
[Thursday 02 September 2010] [08:36:17] <mato> sustrik: ok, so this GIL thing
[Thursday 02 September 2010] [08:36:24] <mato> sustrik: it’s an interesting problem
[Thursday 02 September 2010] [08:37:13] <sustrik> yes?
[Thursday 02 September 2010] [08:37:24] <mato> what i see is that in my python application zmq_close() is getting called, due to Python gc kicking in
[Thursday 02 September 2010] [08:37:57] <mato> however, at the same time, zmq_msg_close() on a method shared with Python is called in the i/o thread
[Thursday 02 September 2010] [08:38:35] <mato> now, zmq_close() calls process_commands() which is waiting for a signal from the i/o thread
[Thursday 02 September 2010] [08:39:00] <mato> but that signal never arrives, because the free_fn() called by the zmq_msg_close() in the i/o thread is trying to acquire the Python GIL
[Thursday 02 September 2010] [08:39:05] <mato> -> deadlock
[Thursday 02 September 2010] [08:39:47] <mato> you follow me?
[Thursday 02 September 2010] [08:40:11] <mato> anyway, i can work around it now by not closing yet more sockets :)
[Thursday 02 September 2010] [08:40:17] <mato> but it’s an interesting case
[Thursday 02 September 2010] [08:40:35] <mato> i will have to make a test case and write it up on the mailing list so that other people can participate
[Thursday 02 September 2010] [08:40:44] <mato> it’s not clear what the solution is
[Thursday 02 September 2010] [08:40:48] <sustrik> i have to draw it down :)
[Thursday 02 September 2010] [08:41:02] <sustrik> mato: yes, please
[Thursday 02 September 2010] [08:41:06] <sustrik> so that it’s not lost
[Thursday 02 September 2010] [08:41:09] <mato> definitely
[Thursday 02 September 2010] [08:41:18] <mato> it needs to be solved, and i suspect the same problem can occur in 2.1
[Thursday 02 September 2010] [08:41:34] <mato> however, it’s possible the solution needs to be done on the pyzmq side
[Thursday 02 September 2010] [08:43:16] <lestrrat> sustrik: wrt EINTR thing, what version should I be expecting it to be implemented? just want to know, cause I’d like to wait my first release of mongrel2 handler until that’s fixed.
[Thursday 02 September 2010] [08:43:35] <lestrrat> don’t want to tell my users to send a SIGKILL to stop their handlers :)
[Thursday 02 September 2010] [08:43:53] <lestrrat> (not in a hurry, just want to get an idea)
[Thursday 02 September 2010] [08:44:02] <sustrik> lestrrat: the problem is that it isn’t backward compatible
[Thursday 02 September 2010] [08:44:07] <sustrik> :|
[Thursday 02 September 2010] [08:44:08] <lestrrat> ah
[Thursday 02 September 2010] [08:44:12] <mato> EINTR again ?
[Thursday 02 September 2010] [08:44:12] <lestrrat> yeah, true
[Thursday 02 September 2010] [08:44:15] <sustrik> yes
[Thursday 02 September 2010] [08:44:34] <sustrik> it turns out that Ctrl+C problem in bindings
[Thursday 02 September 2010] [08:44:41] <sustrik> can be solved by returning EINTR
[Thursday 02 September 2010] [08:44:50] <mato> sustrik: have you verified this?
[Thursday 02 September 2010] [08:45:02] <sustrik> lestrrat did with perl binding
[Thursday 02 September 2010] [08:45:15] <mato> hmm, i think i know why it “solves” the problem
[Thursday 02 September 2010] [08:45:21] <mato> or i can guess...
[Thursday 02 September 2010] [08:45:28] <sustrik> i’m still waiting for brian to verify it with python
[Thursday 02 September 2010] [08:45:29] <lestrrat> I’m willing to accept I’m a bit off :)
[Thursday 02 September 2010] [08:45:48] <sustrik> mato: go on
[Thursday 02 September 2010] [08:46:01] <mato> i reckon what happens is returning EINTR means “the language runtime” wakes up
[Thursday 02 September 2010] [08:46:18] <sustrik> right
[Thursday 02 September 2010] [08:46:35] <mato> depending on said language runtime’s signal handling (out of our control, obviously), the runtime waking up may mean that it processes pending signals
[Thursday 02 September 2010] [08:46:50] <mato> hence the “solution”
[Thursday 02 September 2010] [08:47:06] <sustrik> well, it least the runtime has a chance to process the signal
[Thursday 02 September 2010] [08:47:16] <mato> hmm
[Thursday 02 September 2010] [08:47:20] <sustrik> right now it’s completely stuck
[Thursday 02 September 2010] [08:47:23] <mato> yeah
[Thursday 02 September 2010] [08:47:43] <mato> of course the runtime might be doing something entirely different
[Thursday 02 September 2010] [08:47:50] <mato> so it’s not a guaranteed solution
[Thursday 02 September 2010] [08:47:53] <sustrik> not our fault
[Thursday 02 September 2010] [08:47:56] <mato> just one that might help some people
[Thursday 02 September 2010] [08:48:16] <mato> hmm, except...
[Thursday 02 September 2010] [08:48:19] <sustrik> obviously we cannot give guarantees for broken runtimes
[Thursday 02 September 2010] [08:48:31] <mato> yes, but what is your proposed implementation?
[Thursday 02 September 2010] [08:48:44] <sustrik> forward EINTR to the user
[Thursday 02 September 2010] [08:48:48] <mato> yes, except...
[Thursday 02 September 2010] [08:49:08] <mato> the relationship between EINTR from a syscall and SIGINT is afaik nondeterministic
[Thursday 02 September 2010] [08:49:17] <mato> in other words, you might get an EINTR back from a blocking call
[Thursday 02 September 2010] [08:49:18] <mato> or not
[Thursday 02 September 2010] [08:49:26] <mato> you see where this is heading?
[Thursday 02 September 2010] [08:49:31] <lestrrat> should I explain how perl’s signal handling works? if you guys know what’s up, I’m not an entirely an expert either so I’ll let you guys debate the actual implementation ;)
[Thursday 02 September 2010] [08:49:43] <mato> lestrrat: sure, go for it?
[Thursday 02 September 2010] [08:49:45] <mato> .
[Thursday 02 September 2010] [08:50:56] <lestrrat> well, the important part wrt to all of this is that, when we go down to the C world, Perl’s signal handling is effectively silenced.
[Thursday 02 September 2010] [08:51:12] <lestrrat> (C world, as in binding)
[Thursday 02 September 2010] [08:51:42] <lestrrat> when the binding method is called, things are completely left up to the C/C++ routine, so you can do whatever with the signals
[Thursday 02 September 2010] [08:51:59] <lestrrat> and when zmq’s recv() is being called, perl has absolutely no say there
[Thursday 02 September 2010] [08:52:33] <lestrrat> so from perl’s perspective, all it needs is for recv() to return, and let it know from a return value or errno that there was a signal sent somewhere along the line
[Thursday 02 September 2010] [08:53:06] <lestrrat> then (if need be) the Perl binding can tell the perl interpreter that there was a signal, and let perl’s sig handlers run
[Thursday 02 September 2010] [08:53:24] <lestrrat> at least that’s how I understand it to work.
[Thursday 02 September 2010] [08:53:42] <sustrik> right
[Thursday 02 September 2010] [08:54:03] <mato> ok
[Thursday 02 September 2010] [08:54:18] <mato> i’ve not looked into the perl signal handling
[Thursday 02 September 2010] [08:54:18] <sustrik> what i think mato is saying is that OS does not have to interrupt a blocking call (at C level) when SIGINT happens
[Thursday 02 September 2010] [08:54:24] <mato> yes
[Thursday 02 September 2010] [08:54:32] <lestrrat> right
[Thursday 02 September 2010] [08:54:34] <sustrik> how come that Ctrl+C works then
[Thursday 02 September 2010] [08:54:35] <sustrik> ?
[Thursday 02 September 2010] [08:54:42] <sustrik> for C programs?
[Thursday 02 September 2010] [08:55:01] <sustrik> if main thread was stuck in a blocking call
[Thursday 02 September 2010] [08:55:02] <mato> ah, the thing is multiple threads are involved
[Thursday 02 September 2010] [08:55:08] <mato> this is what makes it tricky
[Thursday 02 September 2010] [08:55:25] <sustrik> but how does it work in C world?
[Thursday 02 September 2010] [08:55:27] <mato> ok, in the simple case
[Thursday 02 September 2010] [08:55:30] <mato> (1 thread)
[Thursday 02 September 2010] [08:55:40] <mato> the OS will invoke the default signal handler
[Thursday 02 September 2010] [08:55:46] <mato> which happens to be “exit the program”
[Thursday 02 September 2010] [08:56:10] <mato> the problem is, if you’re handling signals
[Thursday 02 September 2010] [08:56:12] <sustrik> ah, so no SIGINT
[Thursday 02 September 2010] [08:56:16] <sustrik> just hard exit
[Thursday 02 September 2010] [08:56:18] <mato> yes, you get SIGINT
[Thursday 02 September 2010] [08:56:23] <mato> it’s hard to explain...
[Thursday 02 September 2010] [08:56:32] <sustrik> try
[Thursday 02 September 2010] [08:56:46] * mato needs a whiteboard :)
[Thursday 02 September 2010] [08:56:52] <lestrrat> :)
[Thursday 02 September 2010] [08:56:54] <sustrik> ascii graphics!
[Thursday 02 September 2010] [08:57:12] <mikko> sustrik: it’s possible that i might be in Amsterdam whole of next week
[Thursday 02 September 2010] [08:57:22] <mikko> they are still trying to figure out schedules
[Thursday 02 September 2010] [08:57:28] <sustrik> aha
[Thursday 02 September 2010] [08:57:36] <sustrik> ok, we’ll see
[Thursday 02 September 2010] [08:57:37] <mato> ok, so, case 0: 1 thread, no signal handler: result: on ^C default “exit the program” handler is called by the OS
[Thursday 02 September 2010] [08:58:38] <mato> case 1: 1 thread, and a signal handler in that thread: result: on ^C, threads SIGINT handler is called and when that returns the blocking call it interrupted returns EINTR
[Thursday 02 September 2010] [08:58:50] <mato> it is now up to the program to resolve that EINTR
[Thursday 02 September 2010] [08:58:59] <mato> if it did not exit in the signal handler already, of course
[Thursday 02 September 2010] [08:59:15] <sustrik> ack
[Thursday 02 September 2010] [08:59:27] <mato> now, here’s what i imagine most language runtimes do in a signal handler
[Thursday 02 September 2010] [08:59:45] <mato> and in fact most apps with old-style (no separate thread with sigwait()) signal handlers
[Thursday 02 September 2010] [08:59:51] <mato> they set some flag
[Thursday 02 September 2010] [08:59:54] <mato> that’s all :)
[Thursday 02 September 2010] [09:00:14] <mato> hence, when the call returns EINTR, the app/Perl/whatever checks it’s “was i interrupted” flag
[Thursday 02 September 2010] [09:00:27] <mato> and if it was, deals with that in it’s normal control flow
[Thursday 02 September 2010] [09:00:36] <lestrrat> right
[Thursday 02 September 2010] [09:00:44] <sustrik> ok
[Thursday 02 September 2010] [09:00:46] <mato> now, enter 0mq
[Thursday 02 September 2010] [09:00:54] <mato> the i/o threads ignore all signals
[Thursday 02 September 2010] [09:00:56] * sustrik is thrilled
[Thursday 02 September 2010] [09:01:05] <sustrik> right
[Thursday 02 September 2010] [09:01:06] <mato> they may or may not get an EINTR if the process gets a signal
[Thursday 02 September 2010] [09:01:10] <mato> that is entirely OS-dependent
[Thursday 02 September 2010] [09:01:44] <mato> assuming for a moment that they did, the naive implementation would just return that EINTR, which would bubble up to the app thread logic and something useful might happen
[Thursday 02 September 2010] [09:01:53] <mato> however, that is a bad assumption
[Thursday 02 September 2010] [09:02:19] <mato> so, what to do?
[Thursday 02 September 2010] [09:02:20] <sustrik> wait a sec
[Thursday 02 September 2010] [09:02:28] * mato waiting
[Thursday 02 September 2010] [09:02:31] <sustrik> are you saying that I/O threadsmay get the signal;?
[Thursday 02 September 2010] [09:02:45] <mato> I/O threads will never “get” the signal
[Thursday 02 September 2010] [09:02:49] <sustrik> ok
[Thursday 02 September 2010] [09:03:03] <mato> but depending on your OS, they may get an EINTR back from any random syscall
[Thursday 02 September 2010] [09:03:09] <sustrik> ack
[Thursday 02 September 2010] [09:03:14] <mato> well, not any random, but anything that does something complex
[Thursday 02 September 2010] [09:03:21] <mato> i.e. not getpid() but send() is a candidate
[Thursday 02 September 2010] [09:03:24] <sustrik> ok, but that’s beside the point
[Thursday 02 September 2010] [09:03:31] <mato> it’s not
[Thursday 02 September 2010] [09:03:31] <sustrik> beause we are interested in app threads
[Thursday 02 September 2010] [09:03:35] <sustrik> not i/o threads
[Thursday 02 September 2010] [09:03:53] <sustrik> whether i/o thread gets sigint or not, we don’t care
[Thursday 02 September 2010] [09:04:19] <mato> yes we do, because if you were to be 100% compatible you’d have to “emulate” the behaviour i described in case 1
[Thursday 02 September 2010] [09:04:32] <mato> s/compatible/nice to broken apps/interpreters/
[Thursday 02 September 2010] [09:04:46] <sustrik> wait
[Thursday 02 September 2010] [09:04:54] <sustrik> i/o thread just loops
[Thursday 02 September 2010] [09:04:55] <lestrrat> isn’t thre recv() being called on the app thread?
[Thursday 02 September 2010] [09:05:08] <sustrik> so if it gets EINTR it can just ignore it
[Thursday 02 September 2010] [09:05:25] <mato> hmm, yes
[Thursday 02 September 2010] [09:05:30] <sustrik> lestrrat: that’s my point
[Thursday 02 September 2010] [09:05:49] <mato> ok, right, so you want the app thread side of the API to pass EINTR
[Thursday 02 September 2010] [09:05:54] <sustrik> ack
[Thursday 02 September 2010] [09:05:56] <lestrrat> right
[Thursday 02 September 2010] [09:06:18] <mato> hmm
[Thursday 02 September 2010] [09:06:33] <mato> ok, one way to do it w/o breaking compatibility
[Thursday 02 September 2010] [09:06:36] <mato> with existing code
[Thursday 02 September 2010] [09:06:49] <mato> is a context option
[Thursday 02 September 2010] [09:06:57] <mato> “ZMQ_INTERRUPTIBLE”
[Thursday 02 September 2010] [09:07:09] <sustrik> ok, so it would work?
[Thursday 02 September 2010] [09:07:11] <mato> meaning “API calls will return EINTR if interrupted by a signal”
[Thursday 02 September 2010] [09:07:19] <mato> i think so
[Thursday 02 September 2010] [09:07:24] <sustrik> woohoo
[Thursday 02 September 2010] [09:07:30] <mato> note that there is no change on the i/o thread side
[Thursday 02 September 2010] [09:07:35] <lestrrat> coolness
[Thursday 02 September 2010] [09:07:52] <sustrik> or a compile time option
[Thursday 02 September 2010] [09:07:59] <mato> no, compile time is bad
[Thursday 02 September 2010] [09:08:04] <mato> libzmq.so may be shared
[Thursday 02 September 2010] [09:08:06] <lestrrat> I’d vote for ZMQ_INTERRUPTABLE
[Thursday 02 September 2010] [09:08:17] <sustrik> there are no context-wide options
[Thursday 02 September 2010] [09:08:26] <mato> well, they’ll have to be added then :)
[Thursday 02 September 2010] [09:08:40] <sustrik> but this is a hack
[Thursday 02 September 2010] [09:08:44] <mato> ?
[Thursday 02 September 2010] [09:08:57] <sustrik> the right solution is tu return EINTR
[Thursday 02 September 2010] [09:09:22] <sustrik> we need the option only to stay backward compatible with the original lousy solution
[Thursday 02 September 2010] [09:09:39] <mato> hmm hmm
[Thursday 02 September 2010] [09:09:49] <sustrik> i don’t like changing API because of a hack
[Thursday 02 September 2010] [09:10:01] <mato> sustrik: just a minute, i’m still thinking
[Thursday 02 September 2010] [09:10:06] <lestrrat> I was wondering from the beginning, but does it have to “return” EINTR? is it not enough to keep errno = EINTR ?
[Thursday 02 September 2010] [09:10:24] <sustrik> errno = EINTR
[Thursday 02 September 2010] [09:10:26] <mato> lestrrat: yes, we mean return (-1) with errno = EINTR
[Thursday 02 September 2010] [09:10:33] <lestrrat> ah
[Thursday 02 September 2010] [09:10:39] <sustrik> but still it breaks the API specification
[Thursday 02 September 2010] [09:10:54] <sustrik> EINTR is not described as valid error from recv()
[Thursday 02 September 2010] [09:11:04] <mato> recv() or zmq_recv()?
[Thursday 02 September 2010] [09:11:19] <sustrik> zmq_recv(), sorry
[Thursday 02 September 2010] [09:11:30] <mato> so we’ll have to change that, i think
[Thursday 02 September 2010] [09:11:38] <sustrik> people will hate us
[Thursday 02 September 2010] [09:11:52] <mato> then add ZMQ_INTERRUPTIBLE
[Thursday 02 September 2010] [09:11:57] <sustrik> :)
[Thursday 02 September 2010] [09:12:06] <lestrrat> hey, I’m willing to accept a zmq_recv2() ;P
[Thursday 02 September 2010] [09:12:07] <mato> the thing is, the model of never getting EINTR is actually right if you do your handling properly
[Thursday 02 September 2010] [09:12:28] <mato> kind of
[Thursday 02 September 2010] [09:12:33] <sustrik> shrug
[Thursday 02 September 2010] [09:12:48] <lestrrat> does that mean the “correct” way is for me to install a sighandler in my binding?
[Thursday 02 September 2010] [09:13:08] <mato> lestrrat: that probably won’t help because of the way the interpreter is architected
[Thursday 02 September 2010] [09:13:22] <mato> lestrrat: but yes, that would be the correct way
[Thursday 02 September 2010] [09:13:29] <mato> in an ideal world :)
[Thursday 02 September 2010] [09:13:33] <sustrik> in broader sense i would say: let’s keep with POSIX API
[Thursday 02 September 2010] [09:13:45] <sustrik> trying to outsmart it just causes problems
[Thursday 02 September 2010] [09:13:45] <lestrrat> oh yeah, recv is a currently a loop, ain’t it...
[Thursday 02 September 2010] [09:13:58] <mato> sustrik: well, what this requires then is...
[Thursday 02 September 2010] [09:14:10] <mato> sustrik: a nice thorough explanation by email
[Thursday 02 September 2010] [09:14:16] <mato> sustrik: combined with “sorry, we messed up”
[Thursday 02 September 2010] [09:14:23] <sustrik> we did
[Thursday 02 September 2010] [09:14:27] <mato> sustrik: and it’ll just go into 2.1
[Thursday 02 September 2010] [09:14:37] <mato> eventually, not immediately
[Thursday 02 September 2010] [09:14:58] <mato> after actual verification with at least say Perl, Python, Ruby that it does solve the problem
[Thursday 02 September 2010] [09:15:18] <sustrik> actually, it was brian granger who asked for backward compatibility guarantees :)
[Thursday 02 September 2010] [09:15:27] <lestrrat> would it be acceptable to add a different, aptly named function?
[Thursday 02 September 2010] [09:15:41] <lestrrat> zmq_recv_intr() or whatever.
[Thursday 02 September 2010] [09:15:41] <mato> lestrrat: no, because it involves ALL API calls that can block
[Thursday 02 September 2010] [09:15:53] <lestrrat> hmm
[Thursday 02 September 2010] [09:16:07] <mato> it is actually a mistake on our part, but i didn’t realise until today why it was a mistake
[Thursday 02 September 2010] [09:16:32] <mato> well, we could be anal and say “make your signal handling work like this”, but that’s impossible in the real world
[Thursday 02 September 2010] [09:17:02] <sustrik> ok, let me write an email describing the problem
[Thursday 02 September 2010] [09:17:11] <mato> do you understand it well enough?
[Thursday 02 September 2010] [09:17:21] <mato> this email must not be compressed :-)
[Thursday 02 September 2010] [09:17:24] <sustrik> and asking whether breaking backward compatibility is acceptable in this case
[Thursday 02 September 2010] [09:17:25] <sustrik> no
[Thursday 02 September 2010] [09:17:29] <sustrik> you can write it
[Thursday 02 September 2010] [09:17:44] <sustrik> but it involves saying “i am an idiot, sorry”
[Thursday 02 September 2010] [09:17:49] <mato> i’d like to verify that it actually helps
[Thursday 02 September 2010] [09:18:00] <mato> yes, true, you were the one that defined the behaviour
[Thursday 02 September 2010] [09:18:14] <mato> ok, look, i’ll write the text describing the problem
[Thursday 02 September 2010] [09:18:23] <mato> you can send it out, adding “I’m an idiot”
[Thursday 02 September 2010] [09:18:27] <mato> ok? :)
[Thursday 02 September 2010] [09:18:27] <sustrik> :)
[Thursday 02 September 2010] [09:18:41] <sustrik> ok, let’s first check whether it helps
[Thursday 02 September 2010] [09:18:44] <mato> but before you publicy denounce yourself, it would be nice to check first
[Thursday 02 September 2010] [09:18:46] <mato> precisely
[Thursday 02 September 2010] [09:18:58] <mato> sustrik: i would suggest being very pendantic about this
[Thursday 02 September 2010] [09:19:00] <sustrik> if it turns out that it does not i don’t have to call myself an idiotr
[Thursday 02 September 2010] [09:19:12] <mato> i.e. make three test cases (Perl, Python, Ruby)
[Thursday 02 September 2010] [09:19:17] <sustrik> ok, i can create a topic branch
[Thursday 02 September 2010] [09:19:19] <mato> ensure they hang currently
[Thursday 02 September 2010] [09:19:21] <sustrik> fix it there
[Thursday 02 September 2010] [09:19:24] <mato> i.e. ^C doesn’t work
[Thursday 02 September 2010] [09:19:31] <mato> then make your change
[Thursday 02 September 2010] [09:19:36] <mato> and ensure that all works as expected
[Thursday 02 September 2010] [09:19:55] <mato> sustrik: yes, topic branch, involves doc changes and so on
[Thursday 02 September 2010] [09:19:57] <mato> good idea
[Thursday 02 September 2010] [09:20:11] <sustrik> ok, let me do it
[Thursday 02 September 2010] [09:20:14] <lestrrat> let me know when I can test it :)
[Thursday 02 September 2010] [09:20:19] <mato> sustrik: i will find the magic command for you to email the topic branch patch set around
[Thursday 02 September 2010] [09:20:22] <sustrik> lestrrat: i’ll ping you
[Thursday 02 September 2010] [09:20:30] <sustrik> i’ll ping brian as well
[Thursday 02 September 2010] [09:20:32] <mato> sustrik: so that you can give it to e.g. lestrrat
[Thursday 02 September 2010] [09:20:33] <mato> ja
[Thursday 02 September 2010] [09:20:46] <sustrik> he’s willing to test it, we’ve discussed it in the morning
[Thursday 02 September 2010] [09:20:54] <mato> just email the patches around privately if you don’t want to call yourself a potential idiot just yet :)
[Thursday 02 September 2010] [09:20:56] <sustrik> not sure about ruby
[Thursday 02 September 2010] [09:20:58] <mato> (in public) :)
[Thursday 02 September 2010] [09:21:03] <sustrik> cremes: are you here?
[Thursday 02 September 2010] [09:21:53] <lestrrat> +1 for branch (just easier to pull ;)
[Thursday 02 September 2010] [09:22:13] <sustrik> lestrrat: what’s you email
[Thursday 02 September 2010] [09:22:18] <sustrik> how should i ping you?
[Thursday 02 September 2010] [09:22:24] <sustrik> thx
[Thursday 02 September 2010] [09:22:30] <lestrrat> lestrrat @ github, lestrrat @ twitter
[Thursday 02 September 2010] [09:22:34] <sustrik> ok
[Thursday 02 September 2010] [09:22:34] <mato> lestrrat: shhh... i’m slowly teaching sustrik git
[Thursday 02 September 2010] [09:22:42] <lestrrat> lol
[Thursday 02 September 2010] [09:22:52] <mato> start with local branches
[Thursday 02 September 2010] [09:22:53] <mato> :)
[Thursday 02 September 2010] [09:24:07] <sustrik> ok
[Thursday 02 September 2010] [09:27:28] Quit guido_g has left this server (Quit: Boom...).
[Thursday 02 September 2010] [09:29:01] <mato> ok, i have to concentrate on something else for a bit
[Thursday 02 September 2010] [09:29:12] <mato> ping me if you need me, bbl
[Thursday 02 September 2010] [09:31:39] <sustrik> cya
[Thursday 02 September 2010] [09:53:57] <cremes> sustrik: just got here; what do you need?
[Thursday 02 September 2010] [09:54:25] <sustrik> are you seeing problem with Ctrl+C in Ruby?
[Thursday 02 September 2010] [09:54:46] <sustrik> i mean, application not responding to SIGINT?
[Thursday 02 September 2010] [09:55:10] <sustrik> when stuck in 0mq blocking call?
[Thursday 02 September 2010] [09:55:13] <cremes> yes
[Thursday 02 September 2010] [09:55:24] <sustrik> aha, good
[Thursday 02 September 2010] [09:55:31] <sustrik> we think we’ve found a solution
[Thursday 02 September 2010] [09:55:32] <cremes> i don’t think the ruby signal handler runs when external C code is executing
[Thursday 02 September 2010] [09:55:39] <cremes> really? that’s good
[Thursday 02 September 2010] [09:55:54] <sustrik> would you be willing to test it once i have a fix?
[Thursday 02 September 2010] [09:56:33] <sustrik> the idea is that the blocking calls would return EINTR in case of Ctrl+C
[Thursday 02 September 2010] [09:56:57] <sustrik> then the binding can take care or what happens next
[Thursday 02 September 2010] [09:57:11] <cremes> sure
[Thursday 02 September 2010] [09:57:20] <sustrik> great, i’ll ping you once i have it
[Thursday 02 September 2010] [09:57:25] <cremes> ok
[Thursday 02 September 2010] [10:05:49] <mrm2m> ignore those random and threading they are not used.
[Thursday 02 September 2010] [10:06:41] <mrm2m> the “thread started” shows up, but nothing is sent to the corresponding server.
[Thursday 02 September 2010] [10:09:48] <sustrik> mrm2m: the application exits after sending the message?
[Thursday 02 September 2010] [10:10:00] <mrm2m> right
[Thursday 02 September 2010] [10:10:30] <sustrik> in 2.0.8 all the unsent data are discarded when socket is closed
[Thursday 02 September 2010] [10:10:54] <mrm2m> Uh - ok
[Thursday 02 September 2010] [10:11:19] <sustrik> the semantics is changed in 2.1
[Thursday 02 September 2010] [10:11:39] <sustrik> it’s: block context termination while all data are sent
[Thursday 02 September 2010] [10:12:23] <guido_g> so you can’t exit if something is broken?
[Thursday 02 September 2010] [10:12:44] <sustrik> it’s annoying, i know
[Thursday 02 September 2010] [10:13:02] <sustrik> what i want to do is to add SO_LINGER socket option
[Thursday 02 September 2010] [10:13:12] <lestrrat> would be nice if it was configurable
[Thursday 02 September 2010] [10:13:18] <guido_g> ack
[Thursday 02 September 2010] [10:13:18] <sustrik> with same semantics as with POSIX sockets
[Thursday 02 September 2010] [10:13:42] <sustrik> that should do imo
[Thursday 02 September 2010] [10:16:23] Quit bgranger has left this server (Quit: bgranger).
[Thursday 02 September 2010] [11:07:11] Quit poswald has left this server (Quit: poswald).
[Thursday 02 September 2010] [11:10:07] Quit mrm2m has left this server (Quit: Leaving.).
[Thursday 02 September 2010] [11:11:30] Quit poswald has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [11:25:43] <gavinstark> I am following the PUB/SUB example in the ‘guide’ but I’m wondering how I might setup a way for there to be multiple publishers that each subscriber can receive from. Or do I need independent bind/connects for that?
[Thursday 02 September 2010] [11:27:54] <mikko> gavinstark: the latter
[Thursday 02 September 2010] [11:34:29] <cremes> gavinstark: you also need a forwarder device to aggregate all of the publisher’s messages
[Thursday 02 September 2010] [11:35:01] <cremes> (not strictly true, but i think it’s the cleanest way to set things up)
[Thursday 02 September 2010] [11:42:47] <gavinstark> Do I have to pre-list all the publishers in the forwarder config? It seems like I’d have to have multiple <in> with unique values for each publisher that might appear? Or am I missing something?
[Thursday 02 September 2010] [11:55:00] <sustrik> gavinstark: bind your forwarder devices
[Thursday 02 September 2010] [11:55:12] <sustrik> connect the publishers and subscribers
[Thursday 02 September 2010] [11:57:42] <gavinstark> sustrik, what if I do not know the qty of publishers before hand? Won’t each publisher have to bind uniquely? (
tcp://....:5555,
tcp://....:5556, etc.?)
[Thursday 02 September 2010] [11:59:07] <sustrik> pubishers should _connect_ to the forwared
[Thursday 02 September 2010] [11:59:11] <sustrik> forwarder*
[Thursday 02 September 2010] [12:06:44] <gavinstark> surstrik, ah, ok. I just tried that, having the publisher zmq_connect instead of bind, still not quite working. Here is what I did:
[Thursday 02 September 2010] [12:08:10] <gavinstark> ah, I think I see, I was supposed to “bind” on the “in” entry?
[Thursday 02 September 2010] [12:12:45] <sustrik> gavinstark: forwarder should bind both in and out
[Thursday 02 September 2010] [12:12:47] <zedas> lestrrat: re: mongrel2 handlers blocking, i fixed it by making the zeromq IO threads be 1. I’ve been running it like that for weeks without any problems, so you should be fine.
[Thursday 02 September 2010] [12:13:12] <gavinstark> sustrik: Thanks, working perfectly now.
[Thursday 02 September 2010] [12:13:15] <sustrik> zedas: this is a different issue
[Thursday 02 September 2010] [12:13:50] <sustrik> annoying interactions between language runime, OS signals and 0MQ async architecture
[Thursday 02 September 2010] [12:15:02] <zedas> sustrik: ah. you got a link for me about it?
[Thursday 02 September 2010] [12:15:04] <mato> that’s a very polite way of putting it :-)
[Thursday 02 September 2010] [12:15:54] <sustrik> mato: are you able to summarise it for zed?
[Thursday 02 September 2010] [12:16:00] <mato> yeah
[Thursday 02 September 2010] [12:16:04] * sustrik feels incompetent here
[Thursday 02 September 2010] [12:16:08] <mato> signal handling is fucked :-)
[Thursday 02 September 2010] [12:16:11] <mato> end of summary
[Thursday 02 September 2010] [12:16:32] <sustrik> up to the point
[Thursday 02 September 2010] [12:17:04] <sustrik> basically, it has to do with handling Ctrl+C in interpreted languages
[Thursday 02 September 2010] [12:17:07] <mato> to elaborate on my summary, the issue is with the 0mq API not returning EINTR when API calls are interrupted by a signal in the application thread
[Thursday 02 September 2010] [12:17:09] <sustrik> i’m working on a fix now
[Thursday 02 September 2010] [12:17:23] <mato> zedas: problem is most language runtimes do delayed signal handling
[Thursday 02 September 2010] [12:17:33] <mato> zedas: i.e. handler() just sets some flag
[Thursday 02 September 2010] [12:17:39] <mato> zedas: it doesn’t actually do anything
[Thursday 02 September 2010] [12:17:46] <mato> zedas: flag gets picked up when the runtime wakes up
[Thursday 02 September 2010] [12:18:01] <mato> zedas: but since 0mq calls never return EINTR, runtime never wakes up
[Thursday 02 September 2010] [12:22:32] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Thursday 02 September 2010] [12:25:08] Quit keffo has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [12:26:22] <zedas> mato: ah yes, that’d explain it.
[Thursday 02 September 2010] [12:35:20] <ModusPwnens> hi, i’m back again. Hopefully with a question that’s not as dumb this time. Anyways, does anyone know what would cause the first two bytes of a message to get corrupted?
[Thursday 02 September 2010] [12:39:20] <bgranger> zedas: can you say more about what you have been doing to solve this. I was going to look into the EINTR stuff today.
[Thursday 02 September 2010] [12:40:19] <mato> bgranger: I think we have a solution for the EINTR stuff
[Thursday 02 September 2010] [12:40:27] <sustrik> ModusPwnens: do you have a test program?
[Thursday 02 September 2010] [12:40:46] <bgranger> mato: ?
[Thursday 02 September 2010] [12:40:48] <mato> bgranger: oh, hang on, maybe you mean zed’s issue
[Thursday 02 September 2010] [12:40:56] <bgranger> Is that not the same thing?
[Thursday 02 September 2010] [12:41:07] <mato> as opposed to the general issue which manifests itself as “^C doesn’t work in $RANDOM_LANGUAGE”
[Thursday 02 September 2010] [12:41:09] <bgranger> I missed some of the discussion so I am trying to piece it together
[Thursday 02 September 2010] [12:41:14] <mato> ah
[Thursday 02 September 2010] [12:41:24] <bgranger> Are there 2 different issues?
[Thursday 02 September 2010] [12:41:27] <mato> yeah
[Thursday 02 September 2010] [12:41:42] <mato> zedas has an issue with multiple i/o threads
[Thursday 02 September 2010] [12:41:53] <mato> which may or may not have something (different) to do with EINTR
[Thursday 02 September 2010] [12:42:00] <mato> not clear, haven’t had time to look at that
[Thursday 02 September 2010] [12:42:00] <bgranger> And signals?
[Thursday 02 September 2010] [12:42:03] <mato> signals
[Thursday 02 September 2010] [12:42:11] <mato> The problem is as I described just above.
[Thursday 02 September 2010] [12:42:18] <bgranger> What is the idea about EINTR?
[Thursday 02 September 2010] [12:42:33] <mato> When Martin Sustrik made the original API he decided not to return EINTR from blocking calls
[Thursday 02 September 2010] [12:42:39] <bgranger> I did talk to sustrik late last night (for me) and he pointed me to some code ...
[Thursday 02 September 2010] [12:42:46] <bgranger> Right he showed me that code
[Thursday 02 September 2010] [12:43:01] <mato> Right, except that basically breaks standard signal handling
[Thursday 02 September 2010] [12:43:13] <bgranger> Today I am going to put in a print statement in that logic and try to see if it prints with a SIGINT in the Python bindings.
[Thursday 02 September 2010] [12:43:42] <bgranger> It is still not clear if this will work with the Python bindings, but we will see.
[Thursday 02 September 2010] [12:43:43] <mato> To recap, imagine the simplest case with a C program blocking on some syscall, while at the same time handling say SIGINT.
[Thursday 02 September 2010] [12:43:44] <bgranger> I think it may
[Thursday 02 September 2010] [12:43:49] <ModusPwnens> sustrik: Yeah, it’s happening in my code. I’m not entirely sure why. It seems like it happens when it is sent, as the sending side can properly decode the message. However, the receving side cannot because the first two bytes get corrupted for some reason. I can paste my code to a pastebin if you like.
[Thursday 02 September 2010] [12:44:22] <mato> bgranger: Now, if “handling” SIGINT in this programs case means it just prints “Interrupted!” and exits, then fine.
[Thursday 02 September 2010] [12:44:33] <mato> bgranger: That will work even with the current situation in 0MQ.
[Thursday 02 September 2010] [12:44:53] <bgranger> Not in the python bindings...
[Thursday 02 September 2010] [12:45:10] <mato> bgranger: But, if it instead means that the program just sets some random flag, and then expects to process that flag “later”, it won’t work.
[Thursday 02 September 2010] [12:45:19] <bgranger> Right
[Thursday 02 September 2010] [12:45:27] <mato> bgranger: Which is precisely the Python/Perl (at least) case
[Thursday 02 September 2010] [12:45:32] <bgranger> Which is what Python does...yep
[Thursday 02 September 2010] [12:45:55] <sustrik> ModusPwnens: try it
[Thursday 02 September 2010] [12:45:56] <mato> bgranger: So, the only real solution is that 0MQ API calls return EINTR if they get EINTR back from a blocking system call.
[Thursday 02 September 2010] [12:46:12] <bgranger> Will that happen regardless of what signal handlers have been installed?
[Thursday 02 September 2010] [12:46:19] <mato> Yup
[Thursday 02 September 2010] [12:46:20] <ModusPwnens> sustrik: do you want the entire code or just the functions in question?
[Thursday 02 September 2010] [12:46:36] <mato> bgranger: It’s what the OS does.
[Thursday 02 September 2010] [12:46:40] <bgranger> That would definittely solve our problems then! e would be very happy about that.
[Thursday 02 September 2010] [12:46:49] <sustrik> well, i would like a simple example
[Thursday 02 September 2010] [12:46:57] <sustrik> showing the problem
[Thursday 02 September 2010] [12:47:05] <mato> bgranger: If you do e.g. poll () in C, handle a signal in the same thread, then that poll () will return EINTR
[Thursday 02 September 2010] [12:47:19] <mato> bgranger: by “handle a signal” I mean the “set a flag case”
[Thursday 02 September 2010] [12:47:23] <sustrik> bgranger: i’ll fix it and let you know
[Thursday 02 September 2010] [12:47:24] <bgranger> mato: right OK
[Thursday 02 September 2010] [12:47:39] <sustrik> you can test it with python, others will test with perl and ruby
[Thursday 02 September 2010] [12:47:43] <mato> bgranger: By the way, while you’re here, have you had any more Python/GIL issues?
[Thursday 02 September 2010] [12:47:49] <bgranger> sustrik: can you fix it in the 2.0.8 branch. We are not using trunk yet
[Thursday 02 September 2010] [12:48:09] <sustrik> backwards compatibility :|
[Thursday 02 September 2010] [12:48:11] <bgranger> mato: No we have solved those. It was super sutble to get non-copy send/recv working with the GIL though.
[Thursday 02 September 2010] [12:48:20] <bgranger> mato: But it is working well.
[Thursday 02 September 2010] [12:48:24] <mato> bgranger: I have a really interesting case which looks like one of those...
[Thursday 02 September 2010] [12:48:44] <mato> bgranger: Will send to the mailing list, tomorrow.
[Thursday 02 September 2010] [12:49:06] <ModusPwnens> Hmm.well i don’t really have an example..it’s just happening in the code I have written, so i could give you that if you wanted to see it. It started happening after i used google protobufs to encode into a byte array instead of a string.
[Thursday 02 September 2010] [12:49:11] <bgranger> mato: Ok, I will watch for it. The challenge was getting the ref counts of zmq messages synch’d with those of Python.
[Thursday 02 September 2010] [12:49:30] <mato> bgranger: But the short story is, I see gc trying to close() a socket, while at the same time zmq_free_fn() in a different thread is trying to acquire the GIL in order to decrease the message refcount
[Thursday 02 September 2010] [12:49:39] <mato> bgranger: and the result is deadlock
[Thursday 02 September 2010] [12:50:01] <bgranger> Is this in trunk where sockets can move threads?
[Thursday 02 September 2010] [12:50:08] <mato> bgranger: nope, 2.0.8
[Thursday 02 September 2010] [12:50:22] <bgranger> What language?
[Thursday 02 September 2010] [12:50:27] <mato> Python...
[Thursday 02 September 2010] [12:50:35] <bgranger> What version of pyzmq?
[Thursday 02 September 2010] [12:50:55] <mato> latest-ish, let me check
[Thursday 02 September 2010] [12:51:07] <bgranger> Since Saturday?
[Thursday 02 September 2010] [12:51:10] <mato> ah, no
[Thursday 02 September 2010] [12:51:16] <mato> 2.0.7 pyzmq actually
[Thursday 02 September 2010] [12:51:22] <mato> with 2.0.8 zmq
[Thursday 02 September 2010] [12:51:23] <bgranger> I did a bunch of work then. I release a 2.0.7 stable release and master is now 2.0.8 cmpatible.
[Thursday 02 September 2010] [12:51:58] <bgranger> But, I believe that what you are saying it possible. The 1 problem we have is that the zmq_free_fn does have to acquire the GIL. If that can’t happen, you have trouble.
[Thursday 02 September 2010] [12:52:01] <jonrafkind> has anyone used zmq in a real-time game? mostly I just need low latency
[Thursday 02 September 2010] [12:52:13] <mato> bgranger: that’s precisely what I’m seeing
[Thursday 02 September 2010] [12:52:13] <bgranger> There is nothing we can do to get around this.
[Thursday 02 September 2010] [12:52:32] <bgranger> Can you just hold onto the socket ref to prevent gc?
[Thursday 02 September 2010] [12:52:44] <mato> Yes, but I have transient sockets in this application
[Thursday 02 September 2010] [12:52:49] <mato> So tons of fds get leaked
[Thursday 02 September 2010] [12:52:57] <bgranger> Hmmm, that might be tough
[Thursday 02 September 2010] [12:53:06] <bgranger> What do you mean by that?
[Thursday 02 September 2010] [12:53:22] <mato> Well, it’s precisely my workaround (holding onto the socket refs)
[Thursday 02 September 2010] [12:53:29] <bgranger> Ahh, OK.
[Thursday 02 September 2010] [12:53:45] <mato> But that means the underlying fds hang around, so eventually you’ll run out.
[Thursday 02 September 2010] [12:53:46] <bgranger> Is the socket that is being gc’s the one that is sending the msg though?
[Thursday 02 September 2010] [12:53:53] <bgranger> right, you don’t want that
[Thursday 02 September 2010] [12:54:10] <mato> That’s hard to tell at the moment, but at least you’ve confirmed that this can happen.
[Thursday 02 September 2010] [12:54:18] <mato> So the backtraces I see make sense.
[Thursday 02 September 2010] [12:54:22] <mato> I’ll write it up tomorrow.
[Thursday 02 September 2010] [12:54:39] <sustrik> ModusPwnens: if you want me to look at it, strip it down to the simplest possible example that reproduces the bahviour
[Thursday 02 September 2010] [12:54:53] <bgranger> I can do the following. When a socket send a msg, it can add itself to a list of sockets that the Message hold on to. That way the message can prevent the gc, but when the msg goes away, the socket will as well.
[Thursday 02 September 2010] [12:55:09] <sustrik> ModusPwnens: aren’t you overwriting the buffer you’ve sent to 0MQ?
[Thursday 02 September 2010] [12:55:30] <bgranger> but that might keep socket around longer than you want if you have a message that love a long time.
[Thursday 02 September 2010] [12:55:46] <mato> bgranger: That might be a solution, yes.
[Thursday 02 September 2010] [12:55:54] <mato> bgranger: Anyhow, gc is not supposed to be instant, no?
[Thursday 02 September 2010] [12:56:02] <bgranger> Depends
[Thursday 02 September 2010] [12:56:13] <bgranger> if there are cycles or not.
[Thursday 02 September 2010] [12:56:18] <mato> bgranger: So if the socket hangs around for a bit... does it matter too much? As long as it goes away eventually.
[Thursday 02 September 2010] [12:56:30] <sustrik> jonrafkind: i think there are couple of game devs here, rbraley for example
[Thursday 02 September 2010] [12:56:35] <bgranger> Depends on what “a bit” means
[Thursday 02 September 2010] [12:56:48] <mato> True.
[Thursday 02 September 2010] [12:56:53] <ModusPwnens> Sustrik: Well it’s in a loop and I close the message at the end of the loop, initializing it again at the beginning
[Thursday 02 September 2010] [12:57:07] <mato> bgranger: Let me sleep on it, and write up, this is useful to have on the list
[Thursday 02 September 2010] [12:57:12] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [12:57:20] <mato> bgranger: since other people dealing with GC-based languages may run into similar problems.
[Thursday 02 September 2010] [12:57:22] <bgranger> mato: Great
[Thursday 02 September 2010] [12:57:26] <bgranger> Aboslutely
[Thursday 02 September 2010] [12:57:33] <ModusPwnens> and the data that I am initializing it with is a char * which i free at the end of each iteration too
[Thursday 02 September 2010] [12:57:55] <sustrik> show me the sending code
[Thursday 02 September 2010] [12:58:10] <mato> bgranger: Thanks for your help
[Thursday 02 September 2010] [12:58:25] * mato is off to dinner, see you all later
[Thursday 02 September 2010] [12:58:56] <bgranger> later
[Thursday 02 September 2010] [13:00:11] <ModusPwnens> there’s a lot of debugging stuff in there so i’m sorry that it is messy
[Thursday 02 September 2010] [13:00:11] <sustrik> ModusPwnens: when using zmq_msg_init_data you are passing ownership of the buffer to 0MQ
[Thursday 02 September 2010] [13:00:23] <sustrik> so you have to give it a free function
[Thursday 02 September 2010] [13:00:31] <sustrik> and don’t touch the buffer afterwards
[Thursday 02 September 2010] [13:01:12] <sustrik> if you don’t need zero-copy
[Thursday 02 September 2010] [13:01:22] <sustrik> just init the message using zmq_msg_init_size
[Thursday 02 September 2010] [13:01:29] <sustrik> and copt the data into i
[Thursday 02 September 2010] [13:01:30] <sustrik> it
[Thursday 02 September 2010] [13:01:59] <ModusPwnens> So just don’t use init_data at all?
[Thursday 02 September 2010] [13:02:50] <sustrik> do you need zero-copy?
[Thursday 02 September 2010] [13:03:05] <ModusPwnens> I’m not really sure what that is, so I don’t think so.
[Thursday 02 September 2010] [13:03:14] <sustrik> than don’t use it :)
[Thursday 02 September 2010] [13:03:30] <ModusPwnens> Ok! So just use size to initialize it and then memcpy into it?
[Thursday 02 September 2010] [13:03:36] <sustrik> exactly
[Thursday 02 September 2010] [13:03:42] <ModusPwnens> okie doke. Thanks! I will try that!
[Thursday 02 September 2010] [13:03:48] <ModusPwnens> Sorry for the constant questions :S
[Thursday 02 September 2010] [13:04:29] <sustrik> np
[Thursday 02 September 2010] [13:06:09] <jonrafkind> oh i just realied, imatrix is the same company that made SFL. I use that in my project :p
[Thursday 02 September 2010] [13:33:45] Quit gebi has left this server (Ping timeout: 272 seconds).
[Thursday 02 September 2010] [14:02:40] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [14:19:52] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Thursday 02 September 2010] [14:30:16] Join gebi has joined this channel (
~gebi@84.119.80.149).
[Thursday 02 September 2010] [15:22:39] Quit plq has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:43:34] Quit
snerd_ has left this server (Changing host).
[Thursday 02 September 2010] [15:47:33] Quit snerd has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:47:35] Quit impl has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [15:57:13] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 02 September 2010] [16:27:52] Quit xla has left this server (Quit: leaving).
[Thursday 02 September 2010] [16:29:01] Quit ModusPwnens has left this server.
[Thursday 02 September 2010] [16:50:50] Quit gavinstark has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [16:50:52] <ModusPwnens> Hi guys, I have encountered a problem with the official benchmarking utility.
[Thursday 02 September 2010] [16:51:31] <ModusPwnens> It appears to crash if you enter in a very large number of messages, and I was w ndering if this was supposed to happen and if so, why?
[Thursday 02 September 2010] [16:58:49] Nick
impl_ is now known as impl.
[Thursday 02 September 2010] [17:16:55] <cremes> ModusPwnens: which benchmarking utility and what number did you pass to it?
[Thursday 02 September 2010] [17:17:41] <ModusPwnens> The site has changed so I don’t know where the utility is offhand anymore
[Thursday 02 September 2010] [17:18:09] <cremes> what’s the name of it?
[Thursday 02 September 2010] [17:18:31] <ModusPwnens> actually, it’s in my zeromq folder
[Thursday 02 September 2010] [17:18:35] <ModusPwnens> in the bin
[Thursday 02 September 2010] [17:18:38] <ModusPwnens> remote_thr
[Thursday 02 September 2010] [17:18:52] <cremes> oh, the local_thr/remote_thr pair?
[Thursday 02 September 2010] [17:18:55] <ModusPwnens> ya
[Thursday 02 September 2010] [17:19:17] <cremes> so what arguments did you pass it? (i recommend you pastie the output from your shell along with any displayed error)
[Thursday 02 September 2010] [17:19:32] <ModusPwnens> i passed 50 and 25000000
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> C:UsersDavid DawsonDesktopzeromq-2.0.7zeromq-2.0.7bin>remote_thr.exe tcp:/
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> Assertion failed: end_chunk->next (c:usersdavid dawsondesktopzeromq-2.0.7ze
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> romq-2.0.7srcyqueue.hpp:108)
[Thursday 02 September 2010] [17:20:00] <ModusPwnens> This application has requested the Runtime to terminate it in an unusual way.
[Thursday 02 September 2010] [17:20:01] <ModusPwnens> Please contact the application’s support team for more information.
[Thursday 02 September 2010] [17:20:32] <cremes> did it fail only with the 25 million number or with 50 too?
[Thursday 02 September 2010] [17:20:34] <ModusPwnens> bah, sorry, the command prompt is strange
[Thursday 02 September 2010] [17:20:42] <ModusPwnens> no, it’s just the 25 million
[Thursday 02 September 2010] [17:20:50] <ModusPwnens> if i lower it it works fine
[Thursday 02 September 2010] [17:20:58] <ModusPwnens> but I was just curious if it is supposed to fail that way
[Thursday 02 September 2010] [17:21:07] <cremes> no, you may have found a bug
[Thursday 02 September 2010] [17:21:16] <cremes> but before reporting it, install 2.0.8 and try again
[Thursday 02 September 2010] [17:21:18] <ModusPwnens> ok
[Thursday 02 September 2010] [17:21:22] <cremes> no sense in reporting a bug against an old release
[Thursday 02 September 2010] [17:21:32] <ModusPwnens> true enough
[Thursday 02 September 2010] [17:22:10] <ModusPwnens> ok i have to recompile the source, hold on
[Thursday 02 September 2010] [17:22:34] <cremes> just a note on irc etiquette...
[Thursday 02 September 2010] [17:22:44] <cremes> give as much information as possible...
[Thursday 02 September 2010] [17:23:11] <cremes> don’t paste more than 2 lines directly into the channel; use a pastie srevice like pastie.org or gist.github.com for longer stuff
[Thursday 02 September 2010] [17:23:25] <cremes> tell us the name of the programs involved and the version of the library
[Thursday 02 September 2010] [17:23:43] Quit dos000 has left this server (Read error: Connection reset by peer).
[Thursday 02 September 2010] [17:23:48] <ModusPwnens> Ok. I will note that for the future.
[Thursday 02 September 2010] [17:23:56] <ModusPwnens> thanks!
[Thursday 02 September 2010] [17:23:56] <cremes> if you see a crash, it is never supposed to happen so asking if it is seems a bit silly
[Thursday 02 September 2010] [17:24:06] <cremes> np
[Thursday 02 September 2010] [17:24:09] <ModusPwnens> Well, i guess I meant to say if it was already known
[Thursday 02 September 2010] [17:24:47] <cremes> then search the issues on github; all known bugs are reported and tracked there
[Thursday 02 September 2010] [17:25:08] <ModusPwnens> Hmm, ok. I didn’t know about that..
[Thursday 02 September 2010] [17:25:43] <cremes> and now you know! ;)
[Thursday 02 September 2010] [17:25:54] <ModusPwnens> that would be under the issues section?
[Thursday 02 September 2010] [17:26:12] <cremes> correct
[Thursday 02 September 2010] [17:26:34] <ModusPwnens> Okay. Sorry about that..
[Thursday 02 September 2010] [17:27:51] <cremes> we all had to learn it at some point, so don’t worry about it
[Thursday 02 September 2010] [17:28:13] Quit dos000 has left this server (Ping timeout: 258 seconds).
[Thursday 02 September 2010] [17:29:02] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 02 September 2010] [17:52:01] Quit gavinstark has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [18:03:02] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 02 September 2010] [18:31:17] <ModusPwnens> ok, so it still crashes
[Thursday 02 September 2010] [18:31:22] <ModusPwnens> i will paste the output in as ec
[Thursday 02 September 2010] [18:33:20] <ModusPwnens> I am using windows 7 on both computers
[Thursday 02 September 2010] [18:33:36] <ModusPwnens> The computer that generated the error is 64-bit as well
[Thursday 02 September 2010] [18:34:29] <ModusPwnens> the computer running local is only 32-bit
[Thursday 02 September 2010] [19:43:18] Quit poswald has left this server (Quit: poswald).
[Thursday 02 September 2010] [19:47:56] Quit poswald has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [20:07:47] <bgranger> sustrik: Just played with the EINTR logic.
[Thursday 02 September 2010] [20:31:13] Quit ModusPwnens has left this server.
[Thursday 02 September 2010] [20:35:38] Quit jonrafkind has left this server (Ping timeout: 255 seconds).
[Thursday 02 September 2010] [21:25:39] Quit xrfang has left this server (Remote host closed the connection).
[Thursday 02 September 2010] [21:46:33] <halfie> hi, does zeromq supports building of read-only queues with one writer and multiple readers?
[Thursday 02 September 2010] [21:47:38] Quit rossij has left this server (Ping timeout: 245 seconds).
[Thursday 02 September 2010] [22:10:13] Quit gebi has left this server (Ping timeout: 252 seconds).
[Thursday 02 September 2010] [22:14:20] <halfie> ls
[Thursday 02 September 2010] [22:26:17] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [22:37:15] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:08:08] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:17:00] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:26:55] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:29:33] Quit rbraley has left this server (
*.net
*.split).
[Thursday 02 September 2010] [23:29:33] Quit GeekGod has left this server (
*.net
*.split).
[Thursday 02 September 2010] [23:38:49] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:48:43] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Thursday 02 September 2010] [23:56:19] Quit xrfang has left this server (Remote host closed the connection).
[Friday 03 September 2010] [00:31:22] Quit jsimmons has left this server (Quit: Leaving).
[Friday 03 September 2010] [00:39:25] Quit grahamalot has left this server (Quit: grahamalot).
[Friday 03 September 2010] [00:40:21] Join mikejs has joined this channel (
~me@mikej.st).
[Friday 03 September 2010] [01:51:52] Quit bgranger has left this server (Quit: bgranger).
[Friday 03 September 2010] [01:54:31] <guido_g> g’mornin’ (or whatever :)
[Friday 03 September 2010] [02:20:51] <sustrik> guido_g: thanks, i’ll have a look
[Friday 03 September 2010] [02:20:54] <sustrik> morning
[Friday 03 September 2010] [02:21:01] <sjampoo> I am kinda sad to see that with each update I find it more difficult to navigate around zeromq.??? :(
[Friday 03 September 2010] [02:22:05] <sustrik> sjampoo: i thought pieter reverted it to the original state
[Friday 03 September 2010] [02:22:05] <sustrik> let me see
[Friday 03 September 2010] [02:22:15] <sustrik> you mean the news should be in the right pane?
[Friday 03 September 2010] [02:22:28] <sjampoo> yes, at the least
[Friday 03 September 2010] [02:22:54] <sustrik> if you have any idea what to do further, just shoot
[Friday 03 September 2010] [02:24:21] <sjampoo> first thing would be not to change it every month ;) and I send an email about it months ago to the list. I also do not really like the .com / .org split
[Friday 03 September 2010] [02:24:49] <sjampoo> If the .org site is difficult to understand for new users, just make it more easy to understand
[Friday 03 September 2010] [02:25:04] <sustrik> where would you put the community stuff?
[Friday 03 September 2010] [02:25:20] <sjampoo> I would hide that on a separate but consistent page
[Friday 03 September 2010] [02:25:33] <sustrik> www.zeromq.org/dev say?
[Friday 03 September 2010] [02:25:42] <sjampoo> Yes, something like that.
[Friday 03 September 2010] [02:26:00] <sjampoo> I mean only 0.005% is interested what the actual last edited page is.
[Friday 03 September 2010] [02:26:22] <sjampoo> Most people interested in the community, will most likely just want to see GITHUB or the IssueList
[Friday 03 September 2010] [02:26:35] <sustrik> no mailing list?
[Friday 03 September 2010] [02:26:56] <sjampoo> and the mailinglist
[Friday 03 September 2010] [02:27:06] <sjampoo> But I don’t think that should be on the frontpage
[Friday 03 September 2010] [02:27:40] <sustrik> wouldn’t you mind clicking through a simple page each time to the dev page?
[Friday 03 September 2010] [02:28:20] <sjampoo> No, not if it is at a consistent location
[Friday 03 September 2010] [02:28:25] <sjampoo> I could bookmark it
[Friday 03 September 2010] [02:28:29] <sustrik> right
[Friday 03 September 2010] [02:28:42] <sustrik> in summary:
[Friday 03 September 2010] [02:28:50] <sustrik> 1. move zeromq.com to zeromq.org
[Friday 03 September 2010] [02:28:59] <sustrik> 2. move zeromq.org to zeromq.org/community
[Friday 03 September 2010] [02:29:10] <sustrik> 3. move the news to the right pane
[Friday 03 September 2010] [02:29:29] <sustrik> 4. get rid of recent site changes
[Friday 03 September 2010] [02:29:37] <sustrik> that’s it?
[Friday 03 September 2010] [02:30:15] <sjampoo> I am not completely sure what you mean with moving .com to .org.
[Friday 03 September 2010] [02:30:20] <sjampoo> I am not a big fan of the .com design
[Friday 03 September 2010] [02:30:36] <sustrik> that’s because you are part of community
[Friday 03 September 2010] [02:30:44] <sjampoo> No, no
[Friday 03 September 2010] [02:31:06] <sjampoo> I just don’t think that people landing on that site will get a good idea about what 0mq is
[Friday 03 September 2010] [02:31:20] <sustrik> ok, here’s the use case:
[Friday 03 September 2010] [02:31:21] <sjampoo> It needs a little more meat
[Friday 03 September 2010] [02:31:46] <sustrik> you have a problem and someone advises you 0mq may help to solve it
[Friday 03 September 2010] [02:31:55] <sustrik> you type zeromq into goodle
[Friday 03 September 2010] [02:31:58] <sustrik> google
[Friday 03 September 2010] [02:32:15] <sustrik> which, ideally, would bring you to what’s now zeromq.com
[Friday 03 September 2010] [02:32:27] <sustrik> you are under time stress
[Friday 03 September 2010] [02:32:36] <sustrik> so you cannot browse around much
[Friday 03 September 2010] [02:32:41] <sustrik> what you need to know:
[Friday 03 September 2010] [02:32:56] <sustrik> 1. what is it about (very roughly)
[Friday 03 September 2010] [02:33:00] <sustrik> 2. does it run on my OS
[Friday 03 September 2010] [02:33:09] <sustrik> 3. does it work with my language
[Friday 03 September 2010] [02:33:37] <sustrik> 4. i want to see a simple code example just to make sure it’s not a utter shit
[Friday 03 September 2010] [02:33:49] <sustrik> after that i want a one click download
[Friday 03 September 2010] [02:34:09] <sjampoo> All that is true, but you are missing the biggest yet.
[Friday 03 September 2010] [02:34:15] <sustrik> yes?
[Friday 03 September 2010] [02:34:16] <sjampoo> “Why should I care”
[Friday 03 September 2010] [02:34:27] <sustrik> because i have a problem to solve
[Friday 03 September 2010] [02:34:37] <sustrik> i have to deliver tomorrow
[Friday 03 September 2010] [02:34:52] <sustrik> network part is still missing :(
[Friday 03 September 2010] [02:34:53] <sustrik> etc.
[Friday 03 September 2010] [02:35:11] <sjampoo> Yes, but that isn’t explained currently on zmq.com
[Friday 03 September 2010] [02:35:27] <sustrik> it can be improved obviously
[Friday 03 September 2010] [02:35:39] <sustrik> but the main point is that most people are not browsing for fun
[Friday 03 September 2010] [02:35:44] <sustrik> they are looking for solution
[Friday 03 September 2010] [02:36:16] <sustrik> so what they ask is “would this solve my problem” rather then “what it is good for”
[Friday 03 September 2010] [02:36:24] <sjampoo> Absolutetely. Ok, I would really add a link sidebar on the left side of zeromq.com
[Friday 03 September 2010] [02:36:58] <sustrik> what links?
[Friday 03 September 2010] [02:38:16] <sjampoo> all the ‘content’ links on zeromq.org, links to Github, Blog and Mailinglist
[Friday 03 September 2010] [02:38:40] <sustrik> too confusing imo
[Friday 03 September 2010] [02:39:09] <sustrik> that’s good once you are using the thing
[Friday 03 September 2010] [02:39:34] <sustrik> the zeromq.com is there just to streamline the process up to the dowload phase
[Friday 03 September 2010] [02:40:35] <sustrik> once you’re developing with 0mq you would naturally switch to the community site
[Friday 03 September 2010] [02:41:34] * lestrrat didn’t even know there was a zeromq.com
[Friday 03 September 2010] [02:41:57] <sustrik> it’s new
[Friday 03 September 2010] [02:42:06] <sustrik> created day before yesterdat
[Friday 03 September 2010] [02:43:23] <sustrik> sjampoo: i think you are right about .org, it’s less readable now
[Friday 03 September 2010] [02:43:36] <sustrik> but .com (or however it is called) has it meaning
[Friday 03 September 2010] [02:43:57] <sustrik> althought the user it is aimed at are invisible to you – visible through google analytics though
[Friday 03 September 2010] [02:44:37] <sjampoo> sustrik: I can see what it tries to do, but I am afraid that zeromq.com in its current state will not leave the user with a ‘WOW’ feeling.
[Friday 03 September 2010] [02:44:56] <sustrik> that’s not it goal i think
[Friday 03 September 2010] [02:44:57] <sjampoo> Atleast get some sort of ‘Who is already using zeromq’
[Friday 03 September 2010] [02:45:08] <sustrik> hm
[Friday 03 September 2010] [02:45:23] <sustrik> do you look at that kind of think when inspecting unknown projects?
[Friday 03 September 2010] [02:45:35] <sjampoo> sustrik: absolutely
[Friday 03 September 2010] [02:45:42] <sjampoo> Is it worth my time to investigate
[Friday 03 September 2010] [02:45:48] <sjampoo> will it gain me anything?
[Friday 03 September 2010] [02:45:54] <sjampoo> Or is it something flunky and experimental
[Friday 03 September 2010] [02:46:06] <sustrik> it may be part of “making sure it’s not a shit” step
[Friday 03 September 2010] [02:46:17] <sjampoo> exactly
[Friday 03 September 2010] [02:47:18] <sustrik> what about the “introductions”?
[Friday 03 September 2010] [02:47:23] <sustrik> not good enough?
[Friday 03 September 2010] [02:47:55] <sjampoo> well, I was more thinking about some company profiles that are currently using zeromq
[Friday 03 September 2010] [02:48:52] <sjampoo> And it really should be put on the front page, especially if you take the (no doubt high) bounce rate into account.
[Friday 03 September 2010] [02:49:06] <sustrik> does reading a list of company names you never heard about before make you more happy then quotes?
[Friday 03 September 2010] [02:49:16] <sjampoo> hehe
[Friday 03 September 2010] [02:49:16] <sustrik> (i’m not trolling, i’m really interested)
[Friday 03 September 2010] [02:49:26] <sjampoo> sustrik: interesting question
[Friday 03 September 2010] [02:49:33] <sjampoo> no, I think both are well
[Friday 03 September 2010] [02:49:38] <sjampoo> uh i mean will do
[Friday 03 September 2010] [02:49:48] <sjampoo> And I am pretty sure that it doesn’t really matter who says what
[Friday 03 September 2010] [02:50:07] <sjampoo> that’s why astroturfing is so succesful.
[Friday 03 September 2010] [02:50:18] <sjampoo> We people are so sensitive to what others think
[Friday 03 September 2010] [02:50:46] <sustrik> yeah, but its unlikely to get madonna say anything about 0mq
[Friday 03 September 2010] [02:50:57] <sustrik> so you are still stuck with quotes from people you don’t know
[Friday 03 September 2010] [02:51:19] <sjampoo> exactly, and that doesn’t matter
[Friday 03 September 2010] [02:51:28] <sustrik> yes, quote just sounds well
[Friday 03 September 2010] [02:51:40] <sustrik> because someone invested 10 seconds to write it
[Friday 03 September 2010] [02:52:12] <sustrik> company name is less personal
[Friday 03 September 2010] [02:53:13] <sustrik> now, as for moving it to front page
[Friday 03 September 2010] [02:53:27] <sustrik> do you read endorsements first?
[Friday 03 September 2010] [02:53:47] <sustrik> even before what the project is and whether it will run with your OS/language
[Friday 03 September 2010] [02:54:07] * sustrik is doing it other way round
[Friday 03 September 2010] [02:56:12] <sjampoo> I usually read from top top bottom, but sometime I get distracted by bigger and bolder fonts.
[Friday 03 September 2010] [02:56:26] <sjampoo> and when your lucky I actually click on a link instead of back
[Friday 03 September 2010] [02:56:35] <sustrik> hm
[Friday 03 September 2010] [02:56:53] <sustrik> what i do is a type keywords into google bar
[Friday 03 September 2010] [02:57:03] <sustrik> i get a list of more or less matching projects
[Friday 03 September 2010] [02:57:13] <guido_g> re
[Friday 03 September 2010] [02:57:21] <sustrik> then i look at each one discarding those that won’t fit
[Friday 03 September 2010] [02:57:27] <sjampoo> sustrik: yes, exactly
[Friday 03 September 2010] [02:57:28] <guido_g> ahhh web site issues :)
[Friday 03 September 2010] [02:57:32] <sustrik> i spend say 30 seconds on each page
[Friday 03 September 2010] [02:57:43] <guido_g> could one please reconsider the size of the logo?
[Friday 03 September 2010] [02:57:53] <sustrik> do you want it bigger?
[Friday 03 September 2010] [02:57:55] <sustrik> :)
[Friday 03 September 2010] [02:57:59] <guido_g> when viewed on small screen its taking too much space
[Friday 03 September 2010] [02:58:09] * sustrik makes a note
[Friday 03 September 2010] [02:58:11] <guido_g> sustrik: bad boy! :)
[Friday 03 September 2010] [02:58:28] <guido_g> it’s more than 1/3 of vertical screen space
[Friday 03 September 2010] [02:58:36] <sjampoo> For a page to appear in Google it needs to be indexed with the right terms.
[Friday 03 September 2010] [02:58:38] <sustrik> what’s you screen size?
[Friday 03 September 2010] [02:58:42] <sustrik> just for reference
[Friday 03 September 2010] [02:58:57] <guido_g> and this color makes one aggressive...
[Friday 03 September 2010] [02:59:05] <guido_g> 13.6’‘
[Friday 03 September 2010] [02:59:14] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [02:59:14] <sustrik> i meant in pixels
[Friday 03 September 2010] [02:59:15] <sjampoo> guido_g: ah yes colors
[Friday 03 September 2010] [02:59:19] <sjampoo> lets discuss that next
[Friday 03 September 2010] [02:59:30] * sjampoo grins
[Friday 03 September 2010] [02:59:42] <sjampoo> I actually like the red big logo
[Friday 03 September 2010] [02:59:57] <sustrik> yes, i thought of red logo + shades of grey underneath
[Friday 03 September 2010] [03:00:09] <guido_g> 1280x800
[Friday 03 September 2010] [03:00:15] <sjampoo> it would make a nice tattoo
[Friday 03 September 2010] [03:00:15] <sustrik> it’s kind of dull, but maybe hurts eyes less
[Friday 03 September 2010] [03:00:30] <sustrik> guido_g: thanks
[Friday 03 September 2010] [03:01:06] <sustrik> let’s finish the content thing:
[Friday 03 September 2010] [03:01:08] <guido_g> it’s not that bad when using 1920x1200 on a 27’’ display
[Friday 03 September 2010] [03:01:26] <sustrik> so the front page at zeromq.com is meant to convey, in 30 secs
[Friday 03 September 2010] [03:01:34] <guido_g> but still, takes to much space that could’ve been used for better things
[Friday 03 September 2010] [03:01:39] <sustrik> whether the project is usable in your particular case or not
[Friday 03 September 2010] [03:01:54] <sustrik> guido_g: yes, we can do some experimenting with it
[Friday 03 September 2010] [03:02:37] <guido_g> if it should be for new users, why must i have cookies allowed then?
[Friday 03 September 2010] [03:02:57] <sustrik> what does it do if you don’t?
[Friday 03 September 2010] [03:03:13] <guido_g> it shows a message to activate cookies
[Friday 03 September 2010] [03:03:21] <sustrik> yuck
[Friday 03 September 2010] [03:03:33] * sustrik is not a web person, let me ask pieter
[Friday 03 September 2010] [03:03:33] <guido_g> Can’t proceed, you should accept cookies for this domain.
[Friday 03 September 2010] [03:04:23] <guido_g> i mean, i know mq, so i’ll activate cookies
[Friday 03 September 2010] [03:04:45] <guido_g> but if this would have been the first impression, i’d closed the tab
[Friday 03 September 2010] [03:05:07] <sustrik> right, i am not sure what may be the case. if it’s wikidot requiring the cookies we can move the page elsewhere
[Friday 03 September 2010] [03:05:20] <guido_g> i think it’s wikidot
[Friday 03 September 2010] [03:05:23] <sustrik> it doesn’t have to be a wiki after all
[Friday 03 September 2010] [03:05:42] <guido_g> because you need to allow cross-domain cookies (or how they’re called)
[Friday 03 September 2010] [03:06:01] <sustrik> does that happen with .org as well?
[Friday 03 September 2010] [03:06:16] <guido_g> a community site should be accessible even for users taking their privacy serious
[Friday 03 September 2010] [03:06:25] <guido_g> i’ll take a look
[Friday 03 September 2010] [03:06:46] <guido_g> yes
[Friday 03 September 2010] [03:08:10] <guido_g> btw, wikidot cookies
[Friday 03 September 2010] [03:09:39] <sustrik> hm, i switched off cookies and it still works for me
[Friday 03 September 2010] [03:09:50] <sustrik> what browser are you using?
[Friday 03 September 2010] [03:10:00] <guido_g> btw, i think for the community site we don’t need a large funny picture at the top of the entry page
[Friday 03 September 2010] [03:10:09] <guido_g> opera 10.60
[Friday 03 September 2010] [03:11:58] <sustrik> hm
[Friday 03 September 2010] [03:12:14] <guido_g> i don’t like the sound of “hm” :)
[Friday 03 September 2010] [03:12:52] <sustrik> i was just wondering if i should install opera :)
[Friday 03 September 2010] [03:13:05] <guido_g> hehe
[Friday 03 September 2010] [03:13:09] <sustrik> but let’s rather ask at wikidot
[Friday 03 September 2010] [03:13:14] * sustrik makes another note
[Friday 03 September 2010] [03:13:24] <guido_g> ok, thx for listening
[Friday 03 September 2010] [03:13:47] * guido_g is off to $day-job
[Friday 03 September 2010] [03:13:52] <sustrik> thanks for the feedback
[Friday 03 September 2010] [03:14:01] <guido_g> no problem
[Friday 03 September 2010] [03:14:13] Quit guido_g has left this server (Quit: Boom...).
[Friday 03 September 2010] [03:23:45] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 03 September 2010] [03:36:41] Quit xrfang has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 03 September 2010] [03:39:08] Quit xrfang has left this server (Remote host closed the connection).
[Friday 03 September 2010] [03:47:33] <guido_g> re
[Friday 03 September 2010] [03:52:29] Nick vic is now known as Guest26371.
[Friday 03 September 2010] [04:13:34] <sustrik> guido_g: the article at acm queue is good
[Friday 03 September 2010] [04:13:45] <sustrik> i wanted to write something like this for a long time
[Friday 03 September 2010] [04:13:53] <sustrik> but never had enough time to actually do it
[Friday 03 September 2010] [04:14:25] <guido_g> oh, so another dream died :)
[Friday 03 September 2010] [04:14:30] <
halfie_> hi, can I modify the publish-subscribe model so that the “queue” is read-only?
[Friday 03 September 2010] [04:14:32] <
halfie_> I don
[Friday 03 September 2010] [04:14:53] <guido_g> huh?
[Friday 03 September 2010] [04:14:57] <guido_g> what does that mean?
[Friday 03 September 2010] [04:15:04] <guido_g> SUB sockets are read only...
[Friday 03 September 2010] [04:15:41] <
halfie_> guido_g, I mean that the subscriber shouldn’t remove the message
[Friday 03 September 2010] [04:15:57] <sustrik> you need a database, not a messaging system
[Friday 03 September 2010] [04:16:18] <sustrik> publisher -> INSERT
[Friday 03 September 2010] [04:16:19] <guido_g> ack
[Friday 03 September 2010] [04:16:23] <sustrik> subscriber -> SELECT
[Friday 03 September 2010] [04:17:01] <
halfie_> I see, makes sense.
[Friday 03 September 2010] [04:17:31] <
halfie_> some kind of in-memory database with “queue” semantics would solve the problem
[Friday 03 September 2010] [04:18:11] <guido_g> berkeley db
[Friday 03 September 2010] [04:19:12] <mato> ‘morning
[Friday 03 September 2010] [04:19:50] <
halfie_> guido_g, thanks, I will give it a shot. sustrik: thanks for clearing the situation :)
[Friday 03 September 2010] [04:20:20] <sustrik> :)
[Friday 03 September 2010] [04:20:27] <sustrik> mato: morning
[Friday 03 September 2010] [04:20:34] <sustrik> a question
[Friday 03 September 2010] [04:20:38] <mato> yup?
[Friday 03 September 2010] [04:20:39] <sustrik> a signal question!
[Friday 03 September 2010] [04:20:40] <guido_g> hi mato
[Friday 03 September 2010] [04:20:45] <mato> hi guido
[Friday 03 September 2010] [04:20:51] <sustrik> imagine this:
[Friday 03 September 2010] [04:21:03] <sustrik> binding calls 0mq’s blocking function
[Friday 03 September 2010] [04:21:20] <sustrik> 99% of the time is spent in OS’s blocking call
[Friday 03 September 2010] [04:21:32] <sustrik> 1% of time some 0mq processing is being done
[Friday 03 September 2010] [04:21:46] <sustrik> what if user Ctrl+C’s during the 1%?
[Friday 03 September 2010] [04:21:55] <mato> too bad
[Friday 03 September 2010] [04:21:58] <sustrik> :)
[Friday 03 September 2010] [04:22:02] <mato> it’ll get delivered late
[Friday 03 September 2010] [04:22:06] <mato> nothing you can do about that
[Friday 03 September 2010] [04:22:23] <mato> well, there are various weird hacks you can do about that, but believe me, you don’t want to go there
[Friday 03 September 2010] [04:22:29] <sustrik> it won’t be delivered at all no?
[Friday 03 September 2010] [04:22:50] <sustrik> the handler runs, sets a bit
[Friday 03 September 2010] [04:23:02] <sustrik> there’s no blocking call for OS to interrupt
[Friday 03 September 2010] [04:23:06] <sustrik> so nothing happens
[Friday 03 September 2010] [04:23:11] <mato> hmm
[Friday 03 September 2010] [04:23:16] <sustrik> 0mq then calls a blocking call
[Friday 03 September 2010] [04:23:20] <sustrik> and freezes
[Friday 03 September 2010] [04:23:47] <mato> oh dear
[Friday 03 September 2010] [04:23:50] <mato> yes, you may be right
[Friday 03 September 2010] [04:24:02] <sustrik> the whole EINTR thing seems broken
[Friday 03 September 2010] [04:24:06] <mato> i have always said signal handling is broken
[Friday 03 September 2010] [04:24:10] <mato> to put it very politely
[Friday 03 September 2010] [04:24:16] <sustrik> is there any way to make it work?
[Friday 03 September 2010] [04:24:31] <mato> hmm, well, maybe
[Friday 03 September 2010] [04:24:42] <sustrik> imagine you are omnipotent
[Friday 03 September 2010] [04:24:44] <mato> but it’s so horribly racy i wouldn’t want to go there
[Friday 03 September 2010] [04:24:56] <mato> oh, if i’m omnipotent then 0mq is in kernel space, problem goes away
[Friday 03 September 2010] [04:24:57] <sustrik> you can change the language runtimes etc.
[Friday 03 September 2010] [04:25:08] <mato> ah, less than god (linus), ok...
[Friday 03 September 2010] [04:25:11] <mato> just a sub-god :)
[Friday 03 September 2010] [04:25:22] <sustrik> what i’m asking: is there a canonical way to use EINTR
[Friday 03 September 2010] [04:25:28] <sustrik> something that works?
[Friday 03 September 2010] [04:25:31] <mato> well, there is a canonical way i would do it
[Friday 03 September 2010] [04:25:40] <mato> this is one of the ideas i had yesterday
[Friday 03 September 2010] [04:25:44] <mato> it might not be right
[Friday 03 September 2010] [04:25:48] * sustrik is listening
[Friday 03 September 2010] [04:25:50] <mato> but assuming you can hack the language runtime
[Friday 03 September 2010] [04:26:36] <mato> then what you need to do is chuck a call to a thread-safe “zmq_interrupt()” into the low-level signal handler in the runtime
[Friday 03 September 2010] [04:26:41] <mato> i.e. the bit that “sets a flag”
[Friday 03 September 2010] [04:26:58] <mato> this zmq_interrupt() would cause all application API calls to immediately return with EINTR
[Friday 03 September 2010] [04:27:13] <mato> it’s ugly, but it should work
[Friday 03 September 2010] [04:27:37] <sustrik> tweaking it a bit you end up with chained signal handlers
[Friday 03 September 2010] [04:27:49] <mato> yes, except there’s no API for that
[Friday 03 September 2010] [04:27:53] <sustrik> exactly
[Friday 03 September 2010] [04:27:59] <mato> so you don’t want to go there :)
[Friday 03 September 2010] [04:28:08] <mato> trying to emulate chained signal handlers is madness
[Friday 03 September 2010] [04:28:16] <sustrik> so the whole system is really broken
[Friday 03 September 2010] [04:28:20] <mato> yes
[Friday 03 September 2010] [04:28:46] <sustrik> ok, what i do is froward the EINTR to the binding
[Friday 03 September 2010] [04:28:49] <mato> it doesn’t account for the fact that more than one “entity” inside the application might want to know if a signal arrived
[Friday 03 September 2010] [04:28:54] <sustrik> that would work in99% of cases
[Friday 03 September 2010] [04:29:10] <sustrik> the rest...
[Friday 03 September 2010] [04:29:11] <mato> “forward the EINTR to the binding”? how?
[Friday 03 September 2010] [04:29:11] <sustrik> shrug
[Friday 03 September 2010] [04:29:25] <sustrik> if (errno == EINTR) return -1;
[Friday 03 September 2010] [04:29:30] <mato> oh, that, yes...
[Friday 03 September 2010] [04:29:34] <mato> hmm
[Friday 03 September 2010] [04:29:39] <mato> let me check something
[Friday 03 September 2010] [04:31:34] Join plq has joined this channel (
~plq@78.189.30.218).
[Friday 03 September 2010] [04:34:32] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 03 September 2010] [04:44:26] <mato> sustrik: ok, so, yes, forwarding EINTR is the only way at the moment
[Friday 03 September 2010] [04:44:33] <sustrik> ack
[Friday 03 September 2010] [04:44:41] <sustrik> well, there’s another wat
[Friday 03 September 2010] [04:44:43] <sustrik> way
[Friday 03 September 2010] [04:44:45] <mato> yes?
[Friday 03 September 2010] [04:45:03] <sustrik> on each entry to potentially blocking 0mq function
[Friday 03 September 2010] [04:45:11] <sustrik> store the old signal handler
[Friday 03 September 2010] [04:45:14] <sustrik> install your own
[Friday 03 September 2010] [04:45:15] <sustrik> etc.
[Friday 03 September 2010] [04:45:25] <sustrik> then on exit return it to the original state
[Friday 03 September 2010] [04:45:27] <mato> that’s the naive way, yes
[Friday 03 September 2010] [04:45:29] <mato> except
[Friday 03 September 2010] [04:45:35] <mato> IT WILL NEVER WORK :)
[Friday 03 September 2010] [04:45:41] <sustrik> why so?
[Friday 03 September 2010] [04:46:09] <mato> because, and this is what i just double-checked, all the POSIX docs explicitly say “never, ever, ever, try to mix different signal handling mechanisms and threads”
[Friday 03 September 2010] [04:46:25] <mato> phone, hang on
[Friday 03 September 2010] [04:46:56] <sustrik> not sure what that means, but ok, i don’t want to implement that kind of hack anyway
[Friday 03 September 2010] [04:48:28] <mato> let me explain
[Friday 03 September 2010] [04:48:40] <mato> the problem is it only works if you have 1 thread
[Friday 03 September 2010] [04:48:41] <mato> 1 OS thread
[Friday 03 September 2010] [04:48:45] <mato> and no more
[Friday 03 September 2010] [04:49:18] <mato> if you start doing that kind of stuff it will break any application that might actually trying to be do things the right way in a MT envinronment, which is using a sigwait() thread
[Friday 03 September 2010] [04:49:29] <mato> you have no way of knowing what the application is doing...
[Friday 03 September 2010] [04:49:34] <mato> does that make sense?
[Friday 03 September 2010] [04:50:00] <sustrik> i have no idea how sigwait style interacts with the classic style
[Friday 03 September 2010] [04:50:13] <mato> badly
[Friday 03 September 2010] [04:50:16] <mato> i.e. never do that
[Friday 03 September 2010] [04:50:16] <sustrik> but i don’t really care
[Friday 03 September 2010] [04:50:30] <sustrik> so for now i’ll return EINTR
[Friday 03 September 2010] [04:50:31] <mato> yeah, but if you start mucking with signal handlers then you start caring :)
[Friday 03 September 2010] [04:50:33] <sustrik> when available
[Friday 03 September 2010] [04:50:37] <mato> yes
[Friday 03 September 2010] [04:50:49] <mato> returning EINTR will solve the problem “most of the time”
[Friday 03 September 2010] [04:50:50] <sustrik> so sometimes Ctrl+C won’t work
[Friday 03 September 2010] [04:50:53] Quit jsimmons has left this server (Quit: Leaving).
[Friday 03 September 2010] [04:50:58] <mato> but you can still get signal loss someimes
[Friday 03 September 2010] [04:51:01] <sustrik> however, user can press Ctrl+C again
[Friday 03 September 2010] [04:51:11] <mato> this just needs to be documented, that’s all
[Friday 03 September 2010] [04:51:35] * mato sees the 0MQ FAQ on signal handling becoming into a treatise on the brokenness of signals :)
[Friday 03 September 2010] [04:52:03] <sustrik> yeah
[Friday 03 September 2010] [04:52:11] <mato> sustrik: anyway, can i suggest the following process for this:
[Friday 03 September 2010] [04:52:28] <mato> sustrik: 1. Test the solution, as discussed yesterday.
[Friday 03 September 2010] [04:52:43] <mato> sustrik: 2. Post a patch for review, along with your “I’m (almost) an idiot, sorry”
[Friday 03 September 2010] [04:53:19] <mato> sustrik: 3. I will reply to that describing all I know about the problem so that the binding people can understand what’s going on (since they’re the most affected)
[Friday 03 September 2010] [04:53:56] <mato> sustrik: 4. Some consensus will be reached on whether or not this goes into 2.0.x (I’m not opposed either way, it’s a fix for breakage on our part)
[Friday 03 September 2010] [04:54:30] <mato> sustrik: 5. We may learn something useful from the discussion, and I can also bring up the zmq_interrupt() idea which might give a solution for those language runtimes that want to solve this 100%...
[Friday 03 September 2010] [04:54:41] <mato> sustrik: what do you think?
[Friday 03 September 2010] [04:56:31] <mato> phone call, back in 20mins
[Friday 03 September 2010] [04:56:33] <pieterh> re
[Friday 03 September 2010] [04:56:40] <pieterh> i lost my phone and have been offline
[Friday 03 September 2010] [04:56:42] <pieterh> :-(
[Friday 03 September 2010] [04:58:02] <pieterh> sjampoo: i read your remarks about the website
[Friday 03 September 2010] [04:58:11] <pieterh> they make sense and we’ll evolve this over time
[Friday 03 September 2010] [04:59:50] Quit
gebi_ has left this server (Ping timeout: 272 seconds).
[Friday 03 September 2010] [05:20:39] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 03 September 2010] [05:23:29] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [05:30:36] Quit poswald has left this server (Quit: poswald).
[Friday 03 September 2010] [05:35:23] Quit poswald has left this server (Ping timeout: 260 seconds).
[Friday 03 September 2010] [05:46:21] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Friday 03 September 2010] [05:54:53] <mato> sustrik: re
[Friday 03 September 2010] [05:54:59] <mato> sustrik: are you happy with what i propose?
[Friday 03 September 2010] [05:55:15] <mato> sustrik: it seems like the best way to move forward on the signal handling issues
[Friday 03 September 2010] [07:00:57] <sustrik> mato: yes
[Friday 03 September 2010] [07:01:00] <sustrik> re
[Friday 03 September 2010] [07:01:04] <mato> sustrik: see my long email
[Friday 03 September 2010] [07:01:13] <mato> sustrik: i’ve tried to explain the issue, complex as it is
[Friday 03 September 2010] [07:01:27] <sustrik> ok, i’m on it
[Friday 03 September 2010] [07:08:05] Quit guido_g has left this server (Quit: Boom...).
[Friday 03 September 2010] [07:16:43] Join pieterh has joined this channel (
~ph@217.75.64.166).
[Friday 03 September 2010] [07:30:29] <mato> ok, another big long email sent re the Python GIL stuff
[Friday 03 September 2010] [07:30:44] <mato> somewhat braindump format, hopefully it’s clear enough
[Friday 03 September 2010] [08:12:23] <arnorhs> yo
[Friday 03 September 2010] [08:12:42] <arnorhs> Which programming language has the best zero mq bindings?
[Friday 03 September 2010] [08:12:54] <arnorhs> Which one is the most used / most stable?
[Friday 03 September 2010] [08:15:53] <guido_g> mq itself is programmin in c++, so this should work
[Friday 03 September 2010] [08:16:08] <guido_g> c is delivered with the core, so does work too
[Friday 03 September 2010] [08:16:38] <sustrik> yes, all the bindings use C API underneath
[Friday 03 September 2010] [08:16:42] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [08:17:04] <guido_g> the other bindings are quite stable, if you don’t insinst on using bleeding edge versions
[Friday 03 September 2010] [08:26:55] <arnorhs> ok, thanks
[Friday 03 September 2010] [08:27:03] <arnorhs> I will go for fortran then
[Friday 03 September 2010] [08:27:11] <arnorhs> :)
[Friday 03 September 2010] [08:27:24] <arnorhs> I’m guessing the python bindings are fine?
[Friday 03 September 2010] [08:30:17] <sustrik> yes, i think they are most stable among the whole bunch
[Friday 03 September 2010] [08:51:24] <arnorhs> ok, thanks
[Friday 03 September 2010] [08:53:11] <gebi> are there any plans to add {active, once} mode to the erlang bindings?
[Friday 03 September 2010] [08:53:35] Quit arnorhs has left this server (Remote host closed the connection).
[Friday 03 September 2010] [08:58:54] <sustrik> gebi: what’s once?
[Friday 03 September 2010] [09:23:46] Quit plq has left this server (Remote host closed the connection).
[Friday 03 September 2010] [09:30:37] <gebi> sustrik: only deliver ONE message
[Friday 03 September 2010] [09:31:07] <gebi> with active,true is unuseable because it will fill up your process message queue and make it unusable
[Friday 03 September 2010] [09:31:45] <gebi> active,false is also not ideal because processes block and do not process other messages
[Friday 03 September 2010] [10:31:44] <sustrik> gebi: how does that work
[Friday 03 September 2010] [10:31:50] <sustrik> ok, it delivers one message
[Friday 03 September 2010] [10:31:55] <sustrik> when then?
[Friday 03 September 2010] [10:32:10] <sustrik> you still have to have a way to get next message
[Friday 03 September 2010] [10:32:20] <sustrik> mato: are you here?
[Friday 03 September 2010] [10:35:26] <sustrik> gebi: does it mean there’s at most one message in erlang queue?
[Friday 03 September 2010] [10:35:42] <sustrik> if so, how do you tell erlang to behave that way?
[Friday 03 September 2010] [10:59:09] <gebi> sustrik: it should be a property of the erlang 0mq binding
[Friday 03 September 2010] [11:00:20] <sustrik> :)
[Friday 03 September 2010] [11:00:37] <sustrik> i know, i was just interested whether erlnag as such has such feature
[Friday 03 September 2010] [11:00:41] <gebi> with active, once it just sends delivers a single received datagram into the message queue and stops then
[Friday 03 September 2010] [11:00:42] <sustrik> erlang*
[Friday 03 September 2010] [11:01:03] <sustrik> that’s erlang’s native networking API?
[Friday 03 September 2010] [11:01:49] <gebi> yea it’s a property of gen_tcp
[Friday 03 September 2010] [11:02:16] * sustrik is checking it
[Friday 03 September 2010] [11:03:13] <gebi> you just call inet:setopts(S,[{active,once}]) when you want to get another datagram delivered into your messagequeue
[Friday 03 September 2010] [11:04:25] <sustrik> aha
[Friday 03 September 2010] [11:04:33] <sustrik> how is that different from passive?
[Friday 03 September 2010] [11:04:55] <gebi> what do you mean with passive?
[Friday 03 September 2010] [11:05:11] <sustrik> active=false
[Friday 03 September 2010] [11:05:23] Quit dos000 has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [11:05:23] Quit sachinr has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [11:06:34] <gebi> with active false you have an effectively dead process (all your receiving processes hang on erlzmq:receive)
[Friday 03 September 2010] [11:07:36] <gebi> with all the bad sideeffects
[Friday 03 September 2010] [11:08:20] <sustrik> ah, it blocks threads in erlnag thread pool?
[Friday 03 September 2010] [11:08:25] <sustrik> erlnag*
[Friday 03 September 2010] [11:08:28] <sustrik> erlang*
[Friday 03 September 2010] [11:08:34] <gebi> no erlang processes
[Friday 03 September 2010] [11:09:15] <sustrik> does it matter? if process is up to receiving a message am there’s no message available it hangs anyway
[Friday 03 September 2010] [11:09:55] <gebi> if it “hangs” on receiving a erlang message it is alive, if it hangs on receiving a message from the 0mq binding it is dead
[Friday 03 September 2010] [11:10:23] <sustrik> does “dead” mean that it is ubable to process erlang messages?
[Friday 03 September 2010] [11:10:25] <gebi> meaning it wont process erlang messages till at least _one_ 0mq message gets delivered (and erlzmq:recv returns)
[Friday 03 September 2010] [11:10:32] <sustrik> ok, got it
[Friday 03 September 2010] [11:10:48] <sustrik> makes sense
[Friday 03 September 2010] [11:10:51] <gebi> this _is_ the fastest mode, though most of the time not desireable
[Friday 03 September 2010] [11:11:21] <sustrik> good, i’ll probably have a look at it next week
[Friday 03 September 2010] [11:11:26] <gebi> same with active,true, processing the message-queue in erlang is NOT O(1)
[Friday 03 September 2010] [11:11:39] <sustrik> ack
[Friday 03 September 2010] [11:11:57] <gebi> so it’s kinda ugly to have a big messagequeue, thus active,true is not possibel without another protocol above for flowcontrole
[Friday 03 September 2010] [11:12:00] <gebi> thx :)
[Friday 03 September 2010] [11:12:37] <sustrik> i was actually wondering how the flow control thing can be handled in erlang
[Friday 03 September 2010] [11:12:48] <sustrik> you’ve gave me the answer for free :)
[Friday 03 September 2010] [11:12:53] <gebi> heh, np :)
[Friday 03 September 2010] [11:31:58] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r9e7a9d1 10/ src/zmq.cpp : zmq_poll returns EINTR if signal interrupts it -
http://bit.ly/cWOXej[Friday 03 September 2010] [11:31:59] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * rb059116 10/ (9 files): EINTR is propogated from signaler_t::recv all the way up to the 0MQ API -
http://bit.ly/bOz5Ui[Friday 03 September 2010] [11:32:01] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r63706a9 10/ (doc/zmq_poll.txt doc/zmq_recv.txt doc/zmq_send.txt): documentation modified to reflect the EINTR change -
http://bit.ly/92QIrS[Friday 03 September 2010] [11:51:44] <cremes> sustrik: i get an assertion failure when running the EINTR branch
[Friday 03 September 2010] [11:55:25] <sustrik> cremes: let me see
[Friday 03 September 2010] [12:01:47] Quit
feroz_ has left this server (Quit:
feroz_).
[Friday 03 September 2010] [12:07:10] <sustrik> cremes: try now
[Friday 03 September 2010] [12:08:03] <CIA-20> zeromq2: 03Martin Sustrik 07eintr * r1bc4e9e 10/ src/app_thread.cpp : one more EINTR fix -
http://bit.ly/ciucLd[Friday 03 September 2010] [12:10:07] <cremes> sustrik: works
[Friday 03 September 2010] [12:11:06] <sustrik> you mean Ctrl+C stops the app?
[Friday 03 September 2010] [12:11:12] <cremes> yessir
[Friday 03 September 2010] [12:11:15] <sustrik> woo-hoo!
[Friday 03 September 2010] [12:11:19] <cremes> indeed!
[Friday 03 September 2010] [12:11:30] <sustrik> that’s a bug ~2 years old
[Friday 03 September 2010] [12:11:43] <sustrik> maybe 2 1/2
[Friday 03 September 2010] [12:11:58] <sustrik> until now nobody had any clue how to solve ti
[Friday 03 September 2010] [12:12:01] <cremes> wow, good catch
[Friday 03 September 2010] [12:12:02] <sustrik> great!
[Friday 03 September 2010] [12:12:26] <cremes> is this going into 2.0.9 or 2.1?
[Friday 03 September 2010] [12:12:30] <sustrik> let’s wait for bgranger and lestrrat to confirm it’s working first
[Friday 03 September 2010] [12:12:38] <cremes> sure thing
[Friday 03 September 2010] [12:12:46] <sustrik> i am afraid of pushing it to a stable version
[Friday 03 September 2010] [12:12:51] <cremes> i’ll drop a note to the ML to make my report “official”
[Friday 03 September 2010] [12:13:07] <sustrik> yes, good idea
[Friday 03 September 2010] [12:13:22] <cremes> yeah, i wrote to the ML with my reasoning for this change to go into 2.1 but not 2.0.x
[Friday 03 September 2010] [12:13:30] <sustrik> +1
[Friday 03 September 2010] [12:13:51] <sustrik> at least it’s a motivation for migration to 2.1 :)
[Friday 03 September 2010] [12:16:17] <cremes> oh yeah, i’m looking forward to the thread migration changes too
[Friday 03 September 2010] [12:16:31] <cremes> and the new FD_EVENT stuff
[Friday 03 September 2010] [12:18:49] <sustrik> it’s there to test already
[Friday 03 September 2010] [12:19:00] <sustrik> it may be a bit buggy
[Friday 03 September 2010] [12:19:02] Quit
dos000_ has left this server (Quit: Leaving).
[Friday 03 September 2010] [12:19:03] <sustrik> but still...
[Friday 03 September 2010] [12:19:42] <cremes> sustrik: i’m pretty busy building out a production app using 0mq 2.0.x so i haven’t had any time to test 2.1.x
[Friday 03 September 2010] [12:20:00] <cremes> when you get closer to a release, you should ping the bindings authors and ask us to test
[Friday 03 September 2010] [12:20:06] <sustrik> sure, just updating you on the status
[Friday 03 September 2010] [12:20:12] <cremes> cool, i appreciate it
[Friday 03 September 2010] [12:20:14] <sustrik> we’ll do some testing ourselves
[Friday 03 September 2010] [12:20:19] <sustrik> in the next few weeks
[Friday 03 September 2010] [12:20:28] <sustrik> then it’s binding autohr’s turn
[Friday 03 September 2010] [12:20:45] <cremes> do you have a time frame for the 2.1 release? 2-4 weeks? 4-6 weeks?
[Friday 03 September 2010] [12:22:09] <sustrik> hm, it’s up to mato, he’s going to do the testing
[Friday 03 September 2010] [12:22:30] <sustrik> bet yeah, something between 2 ans 6 weeks i would say
[Friday 03 September 2010] [12:27:05] <cremes> ok
[Friday 03 September 2010] [12:28:10] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [13:14:25] Quit xla has left this server (Quit: leaving).
[Friday 03 September 2010] [13:31:21] <ModusPwnens> So, I updated to 2.0.8 and I still have the problem of the benchmarking programs spitting out errors if you enter in very large parameters
[Friday 03 September 2010] [13:31:23] Quit gebi has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [13:32:07] <ModusPwnens> i’m using windows 7 on both operating systems, but the system running the remote end is 64-bit
[Friday 03 September 2010] [13:32:37] <ModusPwnens> and here is the pastebin with the error emssage and input
[Friday 03 September 2010] [13:35:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [13:40:34] <sustrik> ModusPwnens: throughput test?
[Friday 03 September 2010] [13:40:46] <sustrik> try increasing the sleep period in remote_the
[Friday 03 September 2010] [13:41:02] <sustrik> otherwise it doesn’t have enough time to send all the messages
[Friday 03 September 2010] [13:41:57] <ModusPwnens> Ok, I will try that sustrik
[Friday 03 September 2010] [13:42:08] <sustrik> cremes: nice!
[Friday 03 September 2010] [13:42:24] <sustrik> ha, that’s the guy complaining about the website!
[Friday 03 September 2010] [13:42:27] <sustrik> :)
[Friday 03 September 2010] [13:42:38] <cremes> it’s always good to see people writing articles on our favorite networking library
[Friday 03 September 2010] [13:43:23] <sustrik> yes, given how dull topic the networking is, it’s interesting that people even bother
[Friday 03 September 2010] [13:43:40] <cremes> heh
[Friday 03 September 2010] [13:43:42] <sustrik> btw, the site is back to normal
[Friday 03 September 2010] [13:45:51] <cremes> i see that; btw, i do like the new
*.com site
[Friday 03 September 2010] [13:45:54] <cremes> nice and simple
[Friday 03 September 2010] [13:46:02] Part mikejs has left this channel.
[Friday 03 September 2010] [13:46:19] <sustrik> few minor things to fix still...
[Friday 03 September 2010] [13:46:40] <sustrik> the link in the bottom look like there were 6 of them
[Friday 03 September 2010] [13:46:44] <sustrik> while there are only 3
[Friday 03 September 2010] [13:48:15] <sustrik> etc.
[Friday 03 September 2010] [14:06:46] <ModusPwnens> sustrik: I tried your suggestion and received a different error message
[Friday 03 September 2010] [14:07:21] <sustrik> what error message?
[Friday 03 September 2010] [14:07:51] <ModusPwnens> I just added it to the bottom
[Friday 03 September 2010] [14:08:04] <sustrik> well, not enough memory
[Friday 03 September 2010] [14:08:33] <ModusPwnens> so is that expected then?
[Friday 03 September 2010] [14:08:44] <sustrik> messages are held in memory
[Friday 03 September 2010] [14:09:02] <sustrik> so if you create more than your memory can hold, it’s expected
[Friday 03 September 2010] [14:09:11] <sustrik> what you can do though
[Friday 03 September 2010] [14:09:14] <ModusPwnens> Hmm, that makes sense.
[Friday 03 September 2010] [14:09:19] <sustrik> is to use ZMQ_SWAP socket option
[Friday 03 September 2010] [14:09:28] <sustrik> to offload excessive messages to the disk
[Friday 03 September 2010] [14:10:13] <ModusPwnens> Ok. I will try that.
[Friday 03 September 2010] [14:52:54] <lvh> hey
[Friday 03 September 2010] [14:53:11] <lvh> is there any progress or change regarding the zeromq-exposing-a-fd idea?
[Friday 03 September 2010] [14:53:24] <lvh> AFAIK this is what gevent and twisted are waiting for so we can integrate zeromq.
[Friday 03 September 2010] [14:53:43] <lvh> (I want to try zeromq, but if it won’t work with all of my existing event loop-based stuff, like twisted, well, I basically can’t use it.)
[Friday 03 September 2010] [14:54:39] <sustrik> lvh: it’s in trunk
[Friday 03 September 2010] [14:55:04] <sustrik> it’s a development version so it can be a bit unstable still
[Friday 03 September 2010] [14:55:25] <lvh> sustrik: oh cool, that’s interesting
[Friday 03 September 2010] [14:55:26] <sustrik> but it has ZMQ_FD socket option to get an underlying file descriptor from a 0MQ socket
[Friday 03 September 2010] [14:55:56] <sustrik> and ZMQ_EVENTS option that allows you to check whether socket is ready for readin and/or writing
[Friday 03 September 2010] [14:55:58] <lvh> git://github.com/zeromq/zeromq2.git ?
[Friday 03 September 2010] [14:56:04] <sustrik> yes
[Friday 03 September 2010] [14:56:07] <sustrik> “master”
[Friday 03 September 2010] [14:56:28] <lvh> Wait, why do people need to know the underlying fd
[Friday 03 September 2010] [14:56:46] <sustrik> to poll on it
[Friday 03 September 2010] [14:56:49] <lvh> I thought a zmq socket abstracted away from that and not necessarily have an fd, or not necessarily have exactly one
[Friday 03 September 2010] [14:57:00] <lvh> I’m not sure I understand. Isn’t that zmq’s job still?
[Friday 03 September 2010] [14:57:11] <sustrik> how would you poll on something that’s not a fd?
[Friday 03 September 2010] [14:57:16] <lvh> Is the idea to not use the IO parts in ZMQ?
[Friday 03 September 2010] [14:57:48] <lvh> sustrik: Oh, wait, sorry, I misunderstood what it does.
[Friday 03 September 2010] [14:57:51] <sustrik> the underlying fd is not the networking fd
[Friday 03 September 2010] [14:57:55] <lvh> Right.
[Friday 03 September 2010] [14:57:58] <sustrik> it’s 0MQ’s internal fd
[Friday 03 September 2010] [14:58:05] <lvh> Right, yes, that makes a lot more sense.
[Friday 03 September 2010] [14:58:05] <sustrik> used to communicate with I/O threads
[Friday 03 September 2010] [14:58:48] * sustrik is heading for pub
[Friday 03 September 2010] [15:00:03] <lvh> sustrik: have fun
[Friday 03 September 2010] [15:05:14] Quit lvh has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [15:08:21] Join mikejs has joined this channel (
~me@mikej.st).
[Friday 03 September 2010] [15:20:42] <bgranger> foo = Float(0.00001, config=True)
[Friday 03 September 2010] [15:20:56] Quit keffo has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [15:51:31] Quit GeekGod has left this server (Quit: GeekGod).
[Friday 03 September 2010] [16:32:08] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [16:48:25] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [16:49:07] Quit
mw__ has left this server (Changing host).
[Friday 03 September 2010] [17:34:54] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 03 September 2010] [17:41:26] Quit gebi has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [17:51:13] <troutwine> Does anyone have examples of Google’s Protocol Buffers being used as a serialization format with zeromq? I’ve seen it referenced but cannot find any examples.
[Friday 03 September 2010] [17:59:28] Quit keffo has left this server (Ping timeout: 272 seconds).
[Friday 03 September 2010] [18:06:40] Quit troutwine has left this server (Quit: leaving).
[Friday 03 September 2010] [18:14:11] <erickt> anyone who works on pyzmq online today? I’m having trouble trying to make a fedora rpm for the head of the tree
[Friday 03 September 2010] [18:15:15] <erickt> the commit 2a192b9 seems to have caused fedora 13’s rpmbuild to error out
[Friday 03 September 2010] [18:21:57] Quit sjampoo has left this server (Quit: zzz).
[Friday 03 September 2010] [18:22:32] Quit sjampoo has left this server (Client Quit).
[Friday 03 September 2010] [18:43:55] <bgranger> erickt: Hi I have a few minutes...
[Friday 03 September 2010] [18:44:25] <erickt> I think I found the problem, the enums for the devices is wrong
[Friday 03 September 2010] [18:44:30] <erickt> if I understand cython
[Friday 03 September 2010] [18:44:43] <bgranger> Possibly...
[Friday 03 September 2010] [18:45:11] <bgranger> What do you think the issue it.
[Friday 03 September 2010] [18:45:12] <erickt> ZMQ_QUEUE is defined to 1 in 2.0.8, ZMQ_FORWARDER 2, ZMQ_STREAMER to 3
[Friday 03 September 2010] [18:45:37] <erickt> well i haven’t completely confirmed it, but it seems to cause rpmbuild to fail on f13 I think
[Friday 03 September 2010] [18:45:44] <bgranger> Is that different than 2.0.7?
[Friday 03 September 2010] [18:45:58] Quit __moore__ has left this server (Quit: Leaving).
[Friday 03 September 2010] [18:46:16] <bgranger> Do you have an error msg or traceback?
[Friday 03 September 2010] [18:46:29] <Tasser> what’s a context exaclty?
[Friday 03 September 2010] [18:46:54] <erickt> bgranger: no, that’s part of the problem
[Friday 03 September 2010] [18:47:22] <erickt> it’s dying somewhere deep in a redhat command that doesn’t say really what’s going on
[Friday 03 September 2010] [18:47:31] <bgranger> Ahh. It is possible that the _zmq.c needs to be re-created for 2.0.8, but I would be a bit surprised.
[Friday 03 September 2010] [18:47:35] <bgranger> Can you build by hand?
[Friday 03 September 2010] [18:47:46] <erickt> i just know that in 1770dbb I can build an rpm, but with 2a192b96 i cannot
[Friday 03 September 2010] [18:47:47] <bgranger> That should show any issues outright
[Friday 03 September 2010] [18:47:57] <erickt> yeah it builds fine, and seems to be usable
[Friday 03 September 2010] [18:48:03] <bgranger> Are you buildling against 2.0.8 of zeromq
[Friday 03 September 2010] [18:48:05] <erickt> which is why I’m confused
[Friday 03 September 2010] [18:48:06] <erickt> yeah
[Friday 03 September 2010] [18:48:24] <bgranger> Hmmm, that is odd then. Do you have the script that rpmbuild is using?
[Friday 03 September 2010] [18:48:31] <erickt> Tasser: it’s an object to hold all the threads and underlying bits that zmq needs
[Friday 03 September 2010] [18:48:38] <ModusPwnens> troutwine, are you still here? I can help you!
[Friday 03 September 2010] [18:48:44] <bgranger> I do have to leave in a few minutes, so maybe post to the list if you can’t figure it out.
[Friday 03 September 2010] [18:49:15] <ModusPwnens> Also, I am receiving more errors with the benchmarking utility
[Friday 03 September 2010] [18:49:20] <ModusPwnens> this time not associated with large parameters
[Friday 03 September 2010] [18:49:22] <Tasser> erickt, can you have more than once context? not likely?
[Friday 03 September 2010] [18:49:34] <bgranger> Tasser: multiple contexts are just fine.
[Friday 03 September 2010] [18:49:53] <Tasser> hmm
[Friday 03 September 2010] [18:49:59] <bgranger> Tasser: the general guideline is 1 per library. You can use it to isolate your own zmq stuff from that of other libs.
[Friday 03 September 2010] [18:50:00] <Tasser> I’ll have to play around a bit
[Friday 03 September 2010] [18:50:08] <erickt> that’s the rpm spec and a driver script
[Friday 03 September 2010] [18:51:55] <bgranger> Do you know how far it gets?
[Friday 03 September 2010] [18:52:21] <erickt> it compiles and it’s just determining metadata for the rpm
[Friday 03 September 2010] [18:52:24] <bgranger> erickt: sorry I need to run now, can you email the list. Also, have you talked to T. Spura about this?
[Friday 03 September 2010] [18:52:32] <erickt> nope, just found this
[Friday 03 September 2010] [18:57:32] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [19:02:32] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [19:08:28] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [19:09:07] Quit
mw__ has left this server (Changing host).
[Friday 03 September 2010] [19:40:50] Quit
feroz_ has left this server (Ping timeout: 264 seconds).
[Friday 03 September 2010] [19:41:34] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Friday 03 September 2010] [19:43:15] Quit
mw__ has left this server (Quit:
mw__).
[Friday 03 September 2010] [19:57:13] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [19:57:45] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [20:14:13] Quit jonrafkind has left this server (Read error: Connection reset by peer).
[Friday 03 September 2010] [21:18:32] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [21:19:30] <lestrrat> sustrik: I’m going to be out for the day, so won’t be able to check for another 8hrs or so
[Friday 03 September 2010] [21:19:41] <lestrrat> but will do when I get back
[Friday 03 September 2010] [21:20:28] Nick
AndrewBC_ is now known as AndrewBC.
[Friday 03 September 2010] [21:22:18] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 03 September 2010] [21:46:28] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Friday 03 September 2010] [22:09:41] Quit jsimmons has left this server (Ping timeout: 255 seconds).
[Friday 03 September 2010] [22:46:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 03 September 2010] [23:36:02] Quit bgranger has left this server (Quit: bgranger).
[Friday 03 September 2010] [23:43:39] Quit bgranger has left this server (Client Quit).
[Saturday 04 September 2010] [00:03:29] Quit keffo has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 4 01:30:09 2010
[Saturday 04 September 2010] [01:30:09] Topic The channel topic is “Welcome!”.
[Saturday 04 September 2010] [01:30:15] Mode Channel modes: no messages from outside, topic protection
[Saturday 04 September 2010] [01:30:16] Created This channel was created on 2010-02-05 10:44.
[Saturday 04 September 2010] [02:01:47] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [02:03:12] Join joevandyk has joined this channel (
~joe@98.117.99.141).
[Saturday 04 September 2010] [02:03:38] <joevandyk> hello!
[Saturday 04 September 2010] [02:04:26] <joevandyk> i’m using the ruby zmq gem. i’m using the req/res sockets. When i try to send a message after a client has disconnected and i’ve specified no blocking, the sender just blocks forever.
[Saturday 04 September 2010] [02:13:27] <joevandyk> I think I see the problem. When the service disconnects, the client still tries to send a message, and is waiting for a response.
[Saturday 04 September 2010] [02:13:34] <joevandyk> is there a way around that?
[Saturday 04 September 2010] [02:14:02] <joevandyk> i wonder if this is a bug in the ruby library
[Saturday 04 September 2010] [02:14:37] <lestrrat> sustrik: I sent a message to the mailing list, but I messed my From: field :/
[Saturday 04 September 2010] [02:15:11] <lestrrat> sustrik: anyway, I all I wanted to say was that the eintr branch worked :)
[Saturday 04 September 2010] [02:40:24] Quit bgranger has left this server (Quit: bgranger).
[Saturday 04 September 2010] [02:43:59] Quit jonrafkind has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [03:25:08] Quit keffo has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [03:39:46] Quit guido_g has left this server (Quit: Boom...).
[Saturday 04 September 2010] [03:51:24] <sustrik> lestrrat: great, thanks!
[Saturday 04 September 2010] [03:51:44] <sustrik> joevandyk: it’s the way 0mq works
[Saturday 04 September 2010] [03:52:26] <joevandyk> seems odd that if the service disconnects that the client is hung forever
[Saturday 04 September 2010] [03:52:55] <sustrik> it’ll continue working once service becaomes available
[Saturday 04 September 2010] [03:53:34] <sustrik> if you want to explicitly exit in case your task haven’t been completed in a predefined time, use timeouts
[Saturday 04 September 2010] [03:59:04] <joevandyk> sustrik: hm, i don’t see how it can become unblocked once it’s waiting for a recv
[Saturday 04 September 2010] [03:59:19] <sustrik> use zmq_poll
[Saturday 04 September 2010] [03:59:24] <sustrik> there’s a timout parameter there
[Saturday 04 September 2010] [03:59:41] <joevandyk> sustrik: what i’m not understanding is why the client sends a message when there’s no servers to receive it
[Saturday 04 September 2010] [04:00:13] <sustrik> what does “there’s no server” mean?
[Saturday 04 September 2010] [04:00:43] <sustrik> there’s no such thing in networking
[Saturday 04 September 2010] [04:01:10] <sustrik> what you have otoh is “the transfer haven’t been acknowledged in time”
[Saturday 04 September 2010] [04:01:29] <sustrik> which is the timeout
[Saturday 04 September 2010] [04:02:47] <joevandyk> sorry – when the REP socket disconnects. The REQ client had been happily sending data to the REP. But then the REP disconnected. The REQ tried to send data again to the REP and the send suceeded, but there was nothing to send it to.
[Saturday 04 September 2010] [04:19:08] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 04 September 2010] [04:37:29] Quit joevandyk has left this server (Quit: joevandyk).
[Saturday 04 September 2010] [06:09:07] Quit keffo has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [06:19:40] Quit keffo has left this server (Ping timeout: 258 seconds).
[Saturday 04 September 2010] [06:20:50] Join stephank has joined this channel (
~shteef@84.25.113.14).
[Saturday 04 September 2010] [06:22:11] Nick xla is now known as xla_soundcloud.
[Saturday 04 September 2010] [07:29:29] Nick xla_soundcloud is now known as xla_SC.
[Saturday 04 September 2010] [07:40:58] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [08:14:19] <Tasser> cremes, got me the big picture for zmqmachine?
[Saturday 04 September 2010] [08:24:33] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 04 September 2010] [08:33:10] <mato> re
[Saturday 04 September 2010] [08:33:44] <sustrik> hi
[Saturday 04 September 2010] [08:34:11] <mato> sustrik: list of patches to cherry-pick along with instructions coming up shortly
[Saturday 04 September 2010] [08:34:38] <sustrik> thx
[Saturday 04 September 2010] [08:34:40] <mato> sustrik: just need to do some housekeeping (also delete and recreate FD_SETSIZE issue in tracker since I found that bug, not pieter)
[Saturday 04 September 2010] [08:34:54] <sustrik> ok
[Saturday 04 September 2010] [08:42:42] <mato> ok, i can’t delete it, have commented with the current situation and removed the “maint” label
[Saturday 04 September 2010] [08:44:09] <sustrik> fine
[Saturday 04 September 2010] [08:49:44] <mato> sustrik: mumble... pieter also deleted the “empty” zmq_forwarder/streamer/queue documentation
[Saturday 04 September 2010] [08:49:57] <mato> sustrik: this has packaging implications
[Saturday 04 September 2010] [08:50:05] <mato> sustrik: how would you like me to resolve this?
[Saturday 04 September 2010] [08:50:10] <sustrik> get it back
[Saturday 04 September 2010] [08:50:31] <mato> ok, what about the zmq_device documentation he added?
[Saturday 04 September 2010] [08:50:46] <mato> are devices still experimental in 2.0.x?
[Saturday 04 September 2010] [08:51:34] <mato> or would you prefer to revert his commit entirely?
[Saturday 04 September 2010] [08:51:45] <mato> the problem is it has other stuff mixed up in it
[Saturday 04 September 2010] [08:51:59] <mato> like changes to asciidoc.conf which are half-broken
[Saturday 04 September 2010] [08:52:10] <sustrik> is there any problem with device documentation
[Saturday 04 September 2010] [08:52:29] <mato> i don’t know, but you were keeping it as undocumented
[Saturday 04 September 2010] [08:52:40] <mato> there is a problem with the other parts of the commit
[Saturday 04 September 2010] [08:53:22] <mato> like the asciidoc footer only being present for manpages, not for html
[Saturday 04 September 2010] [08:53:28] <sustrik> what’s easier for you?
[Saturday 04 September 2010] [08:54:22] <mato> less work right now would be just to revert the whole commit
[Saturday 04 September 2010] [08:54:49] <mato> and pieter can resubmit his changes
[Saturday 04 September 2010] [08:55:14] <mato> hmm, maybe
[Saturday 04 September 2010] [08:55:45] <sustrik> it’s up to you
[Saturday 04 September 2010] [09:15:30] Quit xla_SC has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [09:25:42] Quit rgl has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [10:03:23] <CIA-20> zeromq2: 03Martin Lucina 07master * r2673a84 10/ (34 files): (log message trimmed)
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: doc: Update zmq_socket(3) for 2.0.8 API changes
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Added man page for the zmq_device method”
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Added clean target that deletes generated man pages”
[Saturday 04 September 2010] [10:03:24] <CIA-20> zeromq2: Revert “Various changes to documentation project:”
[Saturday 04 September 2010] [10:03:34] <CIA-20> zeromq2: 03Martin Lucina 07maint * ree3444f 10/ doc/zmq_socket.txt : doc: Update zmq_socket(3) for 2.0.8 API changes (+8 more commits...) -
http://bit.ly/aRdJ8z[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: 03Martin Lucina 07master * rf850190 10/ include/zmq.h : zmq.h: Fix typo and use of C99 comment -
http://bit.ly/95mVlM[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: 03Martin Lucina 07master * r51a84c1 10/ (src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:08] <CIA-20> zeromq2: Ensure that 0MQ does not attempt to call select() on more than FD_SETSIZE
[Saturday 04 September 2010] [10:25:09] <CIA-20> zeromq2: 03Martin Lucina 07master * rca17612 10/ (include/zmq.h src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:10] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [10:25:10] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [10:25:11] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:11] <CIA-20> zeromq2: zmq.h: Fix typo and use of C99 comment
[Saturday 04 September 2010] [10:25:12] <CIA-20> zeromq2: Conflicts:
[Saturday 04 September 2010] [10:25:15] <CIA-20> zeromq2: 03Martin Lucina 07maint * rf850190 10/ include/zmq.h : zmq.h: Fix typo and use of C99 comment -
http://bit.ly/95mVlM[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: 03Martin Lucina 07maint * r51a84c1 10/ (src/select.cpp src/zmq.cpp):
[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: zmq::select_t, zmq_poll(): assert if FD_SETSIZE reached
[Saturday 04 September 2010] [10:25:16] <CIA-20> zeromq2: Ensure that 0MQ does not attempt to call select() on more than FD_SETSIZE
[Saturday 04 September 2010] [10:43:08] Quit plq has left this server (Remote host closed the connection).
[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Ivo Danihelka 07maint * rae567be 10/ (AUTHORS src/zmq.cpp): improved null checking in zmq_term -
http://bit.ly/anJS6s[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Jon Dyte 07maint * rc2f3b3b 10/ (src/forwarder.cpp src/queue.cpp src/streamer.cpp): forwarder and streamer devices handle multi-part messages correctly -
http://bit.ly/aRxGoy[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Dhammika Pathirana 07maint * r1022789 10/ src/zmq_decoder.cpp : assert on malformed messages -
http://bit.ly/907fWH[Saturday 04 September 2010] [11:12:33] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rdb7fe85 10/ include/zmq.h : Broken device numbering reverted -
http://bit.ly/cncpxn[Saturday 04 September 2010] [11:19:45] Quit poswald has left this server (Quit: poswald).
[Saturday 04 September 2010] [11:20:13] <CIA-20> zeromq2: 03Ivo Danihelka 07master * rae567be 10/ (AUTHORS src/zmq.cpp): improved null checking in zmq_term -
http://bit.ly/anJS6s[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Jon Dyte 07master * rc2f3b3b 10/ (src/forwarder.cpp src/queue.cpp src/streamer.cpp): forwarder and streamer devices handle multi-part messages correctly -
http://bit.ly/aRxGoy[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Dhammika Pathirana 07master * r1022789 10/ src/zmq_decoder.cpp : assert on malformed messages -
http://bit.ly/907fWH[Saturday 04 September 2010] [11:20:14] <CIA-20> zeromq2: 03Martin Sustrik 07master * rdb7fe85 10/ include/zmq.h : Broken device numbering reverted -
http://bit.ly/cncpxn[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: 03Martin Sustrik 07master * r76f2e5d 10/ include/zmq.h : (log message trimmed)
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: Broken device numbering reverted
[Saturday 04 September 2010] [11:20:24] <CIA-20> zeromq2: assert on malformed messages
[Saturday 04 September 2010] [11:20:25] <CIA-20> zeromq2: forwarder and streamer devices handle multi-part messages correctly
[Saturday 04 September 2010] [11:20:26] <CIA-20> zeromq2: improved null checking in zmq_term
[Saturday 04 September 2010] [11:22:45] Quit xla_SC has left this server (Ping timeout: 260 seconds).
[Saturday 04 September 2010] [11:34:39] <AndrewBC> and could use a fresh pair of eyes.
[Saturday 04 September 2010] [11:36:41] <AndrewBC> I also don’t see anything pertaining to this in the buglist
[Saturday 04 September 2010] [11:46:55] <AndrewBC> That gibberish output is looking more and more sinister, you know
[Saturday 04 September 2010] [11:47:00] <AndrewBC> I wonder what that’s about
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: 03Martin Lucina 07master * ra6d3629 10/ (Makefile.am configure.in ChangeLog): (log message trimmed)
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: build: Generate ChangeLog in ‘make dist’, ZIP automatically
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: Change ‘make dist’ to generate the Git ChangeLog file, that way it doesn’t
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: have to be manually updated nor kept in Git which causes unnecessary work.
[Saturday 04 September 2010] [11:50:21] <CIA-20> zeromq2: Also change ‘make dist’ to invoke ‘dist-zip’ automatically to generate a
[Saturday 04 September 2010] [11:50:22] <CIA-20> zeromq2: ZIP as well as a .tar.gz.
[Saturday 04 September 2010] [11:50:23] <CIA-20> zeromq2: 03Martin Lucina 07master * r32fd916 10/ doc/asciidoc.conf :
[Saturday 04 September 2010] [11:50:24] <CIA-20> zeromq2: doc: Add 0MQ version to XHTML11 backend footer
[Saturday 04 September 2010] [11:50:24] <CIA-20> zeromq2: Thanks to Matt Weinstein for the suggestion. -
http://bit.ly/d1r24H[Saturday 04 September 2010] [11:50:25] <CIA-20> zeromq2: 03Martin Lucina 07master * r1e84519 10/ .gitignore : Update .gitignore -
http://bit.ly/bH7Jhp[Saturday 04 September 2010] [11:50:25] <CIA-20> zeromq2: 03Martin Lucina 07master * rd4c8de5 10/ (5 files in 2 dirs):
[Saturday 04 September 2010] [11:50:26] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [11:50:26] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [11:50:27] <CIA-20> zeromq2: Update .gitignore
[Saturday 04 September 2010] [11:50:27] <CIA-20> zeromq2: doc: Add 0MQ version to XHTML11 backend footer
[Saturday 04 September 2010] [11:54:08] <AndrewBC> Okay, I tracked it down to the zmq_recv() in my client. Perhaps there’s something special in the flags I need to be aware of since it’s on the other side of an XREP
[Saturday 04 September 2010] [11:54:30] Quit gebi has left this server (Read error: Operation timed out).
[Saturday 04 September 2010] [11:54:48] <AndrewBC> So I’m reading the API docs now
[Saturday 04 September 2010] [12:01:34] <AndrewBC> beginning to suspect that I should be checking for multipart messages
[Saturday 04 September 2010] [12:06:26] <sccolbert> Perhaps a simple problem, but I appear to have a mem leak on the SUB side of a PUB/SUB pattern
[Saturday 04 September 2010] [12:06:50] <sccolbert> when I connect the SUB to the PUB, any messages not matching the filter cause wasted memory
[Saturday 04 September 2010] [12:07:30] <sccolbert> a simple test send say, 1 million short messages that dont match any filter on the SUB, will cause the memory usage of the SUB to explode
[Saturday 04 September 2010] [12:07:55] <sccolbert> has anyone else experienced this?
[Saturday 04 September 2010] [12:08:47] <AndrewBC> I assume that since the filtering is done on the SUB side, the messages are indeed being transmitted to the socket on that end
[Saturday 04 September 2010] [12:09:50] <sccolbert> this is true, but their memory is never freed
[Saturday 04 September 2010] [12:10:13] <AndrewBC> Are you using a binding, or what?
[Saturday 04 September 2010] [12:10:22] <sccolbert> PyZMQ
[Saturday 04 September 2010] [12:10:28] <sccolbert> on ubuntu 10.04
[Saturday 04 September 2010] [12:11:00] <sccolbert> so if I just connect the SUB and dont set any filter, the SUB process quickly locks up the system
[Saturday 04 September 2010] [12:11:30] <sccolbert> which I cant see as being related to PyZMQ
[Saturday 04 September 2010] [12:12:15] <AndrewBC> I’m not sure, since I’m not a dev of zmq. It’s supposed to handle the memory just fine, if wishes were horses...
[Saturday 04 September 2010] [12:12:45] <AndrewBC> It could be either, though. All it takes is a circular reference for python to not garbage collect objects
[Saturday 04 September 2010] [12:13:11] <AndrewBC> circular, cyclic, bah
[Saturday 04 September 2010] [12:14:00] <sccolbert> this is true, though I would think that since I havent actually called recv() on the sub (since its filtering all messages) that python isnt even aware a message arrived
[Saturday 04 September 2010] [12:20:40] Quit xla_SC has left this server (Ping timeout: 265 seconds).
[Saturday 04 September 2010] [12:21:46] <AndrewBC> Wish I could help you. I’m banging my head up against a bizzare failed assertion myself
[Saturday 04 September 2010] [12:34:40] Quit xla_SC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [12:38:37] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rebf3089 10/ NEWS : NEWS updated for 2.0.9 -
http://bit.ly/doVZJ4[Saturday 04 September 2010] [12:45:32] Quit AndrewBC has left this server (Ping timeout: 272 seconds).
[Saturday 04 September 2010] [12:56:38] <pieter_hintjens> sccolbert: what version of 0MQ are you using?
[Saturday 04 September 2010] [12:58:09] <sccolbert> pieter_hintjens: 2.0.7
[Saturday 04 September 2010] [12:58:50] <pieter_hintjens> when you stop the publisher does the memory consumption of the subscriber go down again?
[Saturday 04 September 2010] [12:59:03] <sccolbert> hmm havent tried that
[Saturday 04 September 2010] [12:59:05] <sccolbert> gimme a sec
[Saturday 04 September 2010] [13:01:36] <sccolbert> no
[Saturday 04 September 2010] [13:01:50] <sccolbert> killing the publisher does not free the memory in the SUB
[Saturday 04 September 2010] [13:04:10] <pieter_hintjens> hmm... so the next steps are to test with the latest 2.0.9 release
[Saturday 04 September 2010] [13:04:19] <pieter_hintjens> and then make a minimal test case that reproduces this, and log an issue
[Saturday 04 September 2010] [13:04:30] <sccolbert> k, lemme grab the new one along with the new PyZMQ
[Saturday 04 September 2010] [13:04:36] <sccolbert> will only take me a couple minutes
[Saturday 04 September 2010] [13:04:52] <sccolbert> a different question:
[Saturday 04 September 2010] [13:04:54] <pieter_hintjens> if you can make a parallel test case in C or C++ that will help identify whether it’s in PyZMQ or 0MQ
[Saturday 04 September 2010] [13:05:04] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [13:05:04] <sccolbert> sure
[Saturday 04 September 2010] [13:05:14] <pieter_hintjens> shoot...
[Saturday 04 September 2010] [13:05:45] <sccolbert> in a pub/sub pattern, if i send a multipart message, and the first part doesnt match the sub filter, does the sub still recieve the rest of the message?
[Saturday 04 September 2010] [13:06:00] <sccolbert> and then just dump it
[Saturday 04 September 2010] [13:06:06] <pieter_hintjens> i actually added this to the guide today but did not publish it yet
[Saturday 04 September 2010] [13:06:14] <pieter_hintjens> a multipart message is one message
[Saturday 04 September 2010] [13:06:23] <pieter_hintjens> think of the first part as the envelope
[Saturday 04 September 2010] [13:06:32] <pieter_hintjens> it’s very convenient to put the pubsub key there
[Saturday 04 September 2010] [13:06:47] <pieter_hintjens> if the key does not match any subscription the entire message is discarded / ignored
[Saturday 04 September 2010] [13:07:11] <sccolbert> is there any performance improvement to be had with that, over just joining my key to the beggininng of my message?
[Saturday 04 September 2010] [13:07:29] <pieter_hintjens> using a pubsub envelope like this means you can do zerocopy on the data part
[Saturday 04 September 2010] [13:07:41] <sccolbert> ah, great
[Saturday 04 September 2010] [13:07:50] <pieter_hintjens> there is no significant performance cost and you can save on formatting the full message
[Saturday 04 September 2010] [13:08:23] <pieter_hintjens> plus you are guaranteed the key remains separate from the data
[Saturday 04 September 2010] [13:08:29] <pieter_hintjens> it’s a Good Thing
[Saturday 04 September 2010] [13:08:40] <sccolbert> yep
[Saturday 04 September 2010] [13:09:01] <sccolbert> also, in a general pub/sub pattern do you recommend epgm over tcp?
[Saturday 04 September 2010] [13:09:13] <CIA-20> zeromq2: 03Martin Sustrik 07maint * r01c463c 10/ (builds/msvc/platform.hpp configure.in): Version number incremented to 2.0.10 -
http://bit.ly/9vL8xS[Saturday 04 September 2010] [13:09:16] <sccolbert> I wasnt able to find much literature on the pro’s vs cons of each
[Saturday 04 September 2010] [13:10:11] <pieter_hintjens> you’d only use pgm/epgm if you have a performance issue with tcp
[Saturday 04 September 2010] [13:10:23] <pieter_hintjens> there is ongoing work to make tcp pubsub much more efficient for large fanouts
[Saturday 04 September 2010] [13:10:37] <pieter_hintjens> in general use tcp unless you know why you need multicast
[Saturday 04 September 2010] [13:10:38] <sccolbert> cool
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rebf3089 10/ NEWS : NEWS updated for 2.0.9 -
http://bit.ly/doVZJ4[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r01c463c 10/ (builds/msvc/platform.hpp configure.in): Version number incremented to 2.0.10 -
http://bit.ly/9vL8xS[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rb4740c1 10/ NEWS : (log message trimmed)
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: Merge branch ‘maint’
[Saturday 04 September 2010] [13:10:47] <CIA-20> zeromq2: * maint:
[Saturday 04 September 2010] [13:10:48] <CIA-20> zeromq2: Version number incremented to 2.0.10
[Saturday 04 September 2010] [13:10:49] <CIA-20> zeromq2: NEWS updated for 2.0.9
[Saturday 04 September 2010] [13:10:49] <CIA-20> zeromq2: Conflicts:
[Saturday 04 September 2010] [13:10:50] <CIA-20> zeromq2: builds/msvc/platform.hpp
[Saturday 04 September 2010] [13:11:24] <Steve-o> usually tcp up to 4-5 peers is better than multicast for almost all scenarios
[Saturday 04 September 2010] [13:11:35] <sccolbert> good to know
[Saturday 04 September 2010] [13:11:40] <sccolbert> thanks
[Saturday 04 September 2010] [13:13:01] <Steve-o> usually NICs give you some form of TCP acceleration compared to none for UDP multicast
[Saturday 04 September 2010] [13:16:07] <sustrik> uh, 2.0.9 is out
[Saturday 04 September 2010] [13:16:38] * sustrik has enough, goes away to play moonlight sonata
[Saturday 04 September 2010] [13:47:53] <sccolbert> pieter_hintjens: tested new version in C and Python
[Saturday 04 September 2010] [13:47:57] <sccolbert> same issues
[Saturday 04 September 2010] [13:48:37] <sccolbert> where should I log the issue?
[Saturday 04 September 2010] [13:50:29] <sccolbert> ah, ok, it only happens in C when the sub hasnt set any filter
[Saturday 04 September 2010] [13:50:44] <sccolbert> setting a filter that doesnt match anything from the PUB solves the issue
[Saturday 04 September 2010] [13:50:50] <sccolbert> in C at least
[Saturday 04 September 2010] [13:51:23] <sccolbert> i will file this with Brian
[Saturday 04 September 2010] [14:18:12] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 04 September 2010] [14:25:46] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [14:41:13] Nick
AndrewBC_ is now known as AndrewBC.
[Saturday 04 September 2010] [15:01:00] <twcc> Hey all, I’m trying to figure out if zeromq ist the right tool for me. I want to have one “Server” and multiple client, which pushes messages to the server. I can’t find a pattern which fits really good - Request-reply would fit best, but as I understood I have to send a reply. - Is this case so trivial, that I should fall back to standard socket programming?
[Saturday 04 September 2010] [15:11:41] Quit xla_SC has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [15:37:54] <Steve-o> twcc: sounds like PUB/SUB
[Saturday 04 September 2010] [15:43:36] <twcc> Because of the unidirectional manner this would fit, but AFAIK the Publisher starts the socket and the subscriber has to connect?! - This could run into problems with a changing number of publisher, or can I connect them to one socket, and zmq handles this?
[Saturday 04 September 2010] [15:53:17] <Steve-o> You can connect multiple to one zmq socket as detailed in zmq_socket(3)
[Saturday 04 September 2010] [15:54:40] <Steve-o> there is currently no bus topology socket pattern
[Saturday 04 September 2010] [15:55:13] <Steve-o> you can jump down to raw PGM for that
[Saturday 04 September 2010] [15:57:09] <twcc> maybe i’m stuck to much to socket programming ... - so I can bind multiple publisher to the same socket/port and have one subscriber which processes the messages? That would be great and solves all my problems :-)
[Saturday 04 September 2010] [16:01:35] <kooroo> hey folks, I’m having trouble compiling zeromq on rh5 x86_64
[Saturday 04 September 2010] [16:01:52] <kooroo> .libs/libzmq_la-txwi.o: relocation R_X86_64_PC32 against
`pgm_rs_create’ can not be used when making a shared object; recompile with -fPIC is what I’m getting.
[Saturday 04 September 2010] [16:02:00] <kooroo> has anyone come across this before?
[Saturday 04 September 2010] [16:02:11] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 04 September 2010] [16:03:06] <Steve-o> twcc: for each subscriber you call zmq_connect for each publisher on one zmq socket
[Saturday 04 September 2010] [16:04:49] <Steve-o> kooroo: it is a bug in RHEL GCC, you can disable PGM or there is a flag you can set to disable visibility on the PGM internal API
[Saturday 04 September 2010] [16:07:08] <kooroo> Steve-o, is this the -DPGM_GNUC_INTERNAL flag?
[Saturday 04 September 2010] [16:07:29] <Steve-o> kooroo: yes, it needs to be “-DPGM_GNUC_INTERNAL=” for broken GCC, default “-DPGM_GNUC_INTERNAL=G_GNUC_INTERNAL”
[Saturday 04 September 2010] [16:08:36] <Steve-o> I think someone found a backported GCC version that works too on the list before
[Saturday 04 September 2010] [16:10:45] <kooroo> Steve-o: do you know offhand if upgrading gcc works?
[Saturday 04 September 2010] [16:11:07] <Steve-o> 4.1.2 is the affected version
[Saturday 04 September 2010] [16:12:36] <kooroo> Steve-o: thanks, Imma upgrade gcc and try it again.
[Saturday 04 September 2010] [16:12:54] <twcc> Hm, maybe I explained it bad - I have a changing number of processes (publisher) which have to send messages to one sink (daemon/subscriber), which handles these messages. A topologie where the daemon/subscriber has to connect to the publisher isn’t feasible. - So the client/server application would fit best, except that the server has to response to a message.
[Saturday 04 September 2010] [16:13:50] <kooroo> twcc, could you have them all subscribe to the same multicast stream and only the subscriber react to messages?
[Saturday 04 September 2010] [16:14:15] <kooroo> not the most efficient way, I admit.
[Saturday 04 September 2010] [16:17:35] <twcc> hm, funny idea, didn’t thought of it
[Saturday 04 September 2010] [16:19:05] <twcc> multicast is only possible on network connections?, because all the processes are running on the same computer - so i would like to use ipc
[Saturday 04 September 2010] [16:24:02] <Steve-o> multicast requires a network for reliable delivery, it’s pretty useless for ipc.
[Saturday 04 September 2010] [16:30:35] <Steve-o> I think as IPC is currently implemented using local sockets you would have to add a shared memory transport for localhost broadcast
[Saturday 04 September 2010] [16:32:58] <twcc> I think the multicast idea would only be a workaround - I would like to have a non-blocking “Request-reply” without reply pattern, but I can’t figure out, how to realize it with zeromq
[Saturday 04 September 2010] [16:36:14] <twcc> Thank you for your help - I will try if the request-reply pattern satisfy the requirements and handle the non-blocking by myself
[Saturday 04 September 2010] [16:43:48] <Steve-o> post a message on the list too, it sounds like a good idea for v3, I think 29West have a broadcast IPC transport too
[Saturday 04 September 2010] [16:50:21] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [16:54:11] Part twcc has left this channel.
[Saturday 04 September 2010] [16:58:57] Quit xla_SC has left this server (Quit: leaving).
[Saturday 04 September 2010] [17:55:35] Quit sccolbert has left this server (Remote host closed the connection).
[Saturday 04 September 2010] [18:23:00] Quit Odizes has left this server (Quit: Saliendo).
[Saturday 04 September 2010] [18:37:19] <DasIch> (obviously what i really did was a bit more complex but that what it boils down to)
[Saturday 04 September 2010] [18:40:56] <lestrrat> sustrik: so the eintr thins is going to be in 2.1?
[Saturday 04 September 2010] [19:01:44] <lestrrat> sustrik: And the ZMQ_FD stuff to get at the file descriptor – is that going to be in 2.1 as well?
[Saturday 04 September 2010] [19:07:22] Quit bgranger has left this server (Quit: bgranger).
[Saturday 04 September 2010] [19:13:20] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Saturday 04 September 2010] [20:01:46] Quit Guest26371 has left this server (Quit: Page closed).
[Saturday 04 September 2010] [20:15:23] Quit DasIch has left this server (Ping timeout: 255 seconds).
[Saturday 04 September 2010] [21:13:32] Quit andrewvc has left this server (Quit: andrewvc).
[Saturday 04 September 2010] [21:45:06] Quit andrewvc has left this server (Quit: andrewvc).
[Saturday 04 September 2010] [21:49:55] Quit xla has left this server (Quit: leaving).
[Saturday 04 September 2010] [22:12:10] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Saturday 04 September 2010] [22:34:34] Quit plq has left this server (Quit: Leaving).
[Saturday 04 September 2010] [22:43:18] Quit jonrafkind has left this server (Ping timeout: 240 seconds).
[Saturday 04 September 2010] [23:15:29] Quit sd88g93 has left this server (Quit: Ex-Chat).
[Saturday 04 September 2010] [23:21:28] Quit jonrafkind has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [01:40:12] <sustrik> lestrrat: both are going to be in 2.1
[Sunday 05 September 2010] [01:40:45] <sustrik> ZMQ_FD as already there
[Sunday 05 September 2010] [01:41:00] <sustrik> i’ll write an email about EINTR & 2.1 today
[Sunday 05 September 2010] [02:00:28] <sustrik> DasIch: That’s a bug in OpenPGM library
[Sunday 05 September 2010] [02:00:53] <sustrik> if you don’t need PGM, just compile 0MQ without it
[Sunday 05 September 2010] [02:01:36] <sustrik> it have been fixed in newer OpenPGM releases
[Sunday 05 September 2010] [02:01:44] <sustrik> but haven’t got into 0MQ as for now
[Sunday 05 September 2010] [02:01:59] <sustrik> anway, creating multiple contexts is pretty useless
[Sunday 05 September 2010] [02:02:08] <sustrik> why do you do that?
[Sunday 05 September 2010] [02:32:34] <lestrrat> sustrik: by “it’s already there” you mean the github repo, right?
[Sunday 05 September 2010] [02:34:06] <sustrik> yes
[Sunday 05 September 2010] [02:34:09] <sustrik> the master branch
[Sunday 05 September 2010] [02:34:33] <sustrik> EINTR on eintr branch, FD & EVENTS on master branch
[Sunday 05 September 2010] [02:35:21] <sustrik> if i have some spare time today, i’ll merge EINTR fix to master as well
[Sunday 05 September 2010] [02:57:14] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 05 September 2010] [03:29:53] Nick
AndrewBC_ is now known as AndrewBC.
[Sunday 05 September 2010] [03:57:15] Topic The channel topic is “Welcome!”.
[Sunday 05 September 2010] [03:57:21] Mode Channel modes: no messages from outside, topic protection
[Sunday 05 September 2010] [03:57:21] Created This channel was created on 2010-02-05 10:44.
[Sunday 05 September 2010] [03:57:29] Quit travlr has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [04:17:47] Quit kooroo has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [04:17:47] Nick kooroox2 is now known as kooroo.
[Sunday 05 September 2010] [04:39:40] <pieter_hintjens> sustrik: hi
[Sunday 05 September 2010] [05:50:29] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 05 September 2010] [05:54:31] <sustrik> pieter_hintjens: hi
[Sunday 05 September 2010] [05:57:29] <pieter_hintjens> hi martin... i was wondering about different ways to submit patches
[Sunday 05 September 2010] [05:57:47] <pieter_hintjens> people are using at least three different routes
[Sunday 05 September 2010] [05:57:59] <pieter_hintjens> patches to zeromq-dev, issues, and pull requests
[Sunday 05 September 2010] [05:58:48] <sustrik> i am using patches from the mailing list
[Sunday 05 September 2010] [05:58:56] <sustrik> are there any pending patches anywhere else?
[Sunday 05 September 2010] [05:59:05] <pieter_hintjens> i saw one pull request on the ML today
[Sunday 05 September 2010] [05:59:22] <sustrik> yup
[Sunday 05 September 2010] [05:59:35] <sustrik> that’s a packaging script thing, it’s up to mato to manage that one
[Sunday 05 September 2010] [05:59:47] * sustrik has no idea how packaging works
[Sunday 05 September 2010] [06:00:04] <pieter_hintjens> :-) it’s more of a consistency thing than anything else
[Sunday 05 September 2010] [06:00:27] <pieter_hintjens> ok, totally different thing, i have a trivial inproc example that refuses to work
[Sunday 05 September 2010] [06:00:39] <sustrik> can you paste is somewhere?
[Sunday 05 September 2010] [06:00:51] <pieter_hintjens> yes, i raised an issue (62) with an example and analysis
[Sunday 05 September 2010] [06:01:00] <sustrik> let me see
[Sunday 05 September 2010] [06:01:56] <pieter_hintjens> i’ve spent several hours trying different things, ended up doing a sleep... :-/
[Sunday 05 September 2010] [06:03:27] <sustrik> i would say it’s the inproc reconnect thing
[Sunday 05 September 2010] [06:03:40] <sustrik> inproc reconnect is still unimplemented
[Sunday 05 September 2010] [06:03:58] <sustrik> thus doing connect before bind does not work
[Sunday 05 September 2010] [06:04:34] <sustrik> so, if step1 connects before step2 binds
[Sunday 05 September 2010] [06:04:38] <sustrik> :|
[Sunday 05 September 2010] [06:04:39] <pieter_hintjens> ah, it’s not just for PAIR sockets
[Sunday 05 September 2010] [06:05:08] <sustrik> it’s inproc as well
[Sunday 05 September 2010] [06:05:18] <pieter_hintjens> with PUSH/PULL it works a little better, say 90% of the time
[Sunday 05 September 2010] [06:05:19] <sustrik> it’s not that visible as inproc connection cannot break
[Sunday 05 September 2010] [06:05:25] <pieter_hintjens> with PAIR it dies almost immediately
[Sunday 05 September 2010] [06:05:35] <sustrik> it’s a timing issue
[Sunday 05 September 2010] [06:05:44] <pieter_hintjens> ok, if this is a known restriction that’s fine, I can document it
[Sunday 05 September 2010] [06:06:19] <sustrik> Connecting a socket
[Sunday 05 September 2010] [06:06:19] <sustrik> When connecting a socket to a peer address using zmq_connect() with the inproc transport, the endpoint shall be interpreted as an arbitrary string identifying the name to connect to. The name
[Sunday 05 September 2010] [06:06:19] <sustrik> must have been previously created by assigning it to at least one socket within the same 0MQ context as the socket being connected.
[Sunday 05 September 2010] [06:06:19] <pieter_hintjens> there is no technical reason it could not work, right?
[Sunday 05 September 2010] [06:06:25] <sustrik> no
[Sunday 05 September 2010] [06:06:26] <pieter_hintjens> hmm, ok
[Sunday 05 September 2010] [06:06:32] <sustrik> just not implemented
[Sunday 05 September 2010] [06:06:39] <pieter_hintjens> excellent, that is all I need
[Sunday 05 September 2010] [06:06:44] <pieter_hintjens> i’ll close the issue
[Sunday 05 September 2010] [06:06:47] <pieter_hintjens> thx
[Sunday 05 September 2010] [06:06:50] <sustrik> np
[Sunday 05 September 2010] [06:08:10] <pieter_hintjens> “ZeroMQ is quickly becoming an even bigger hammer in the premature optimization planet of Newbo-Thumbia.”
[Sunday 05 September 2010] [06:08:24] <pieter_hintjens> vs. “Then they’d better find a new rite of passage, because ZeroMQ is turning into a category killer.”
[Sunday 05 September 2010] [06:08:36] <sustrik> :)
[Sunday 05 September 2010] [06:08:53] <pieter_hintjens> that is an excellent discussion...
[Sunday 05 September 2010] [06:09:00] <sustrik> yes, i liked it
[Sunday 05 September 2010] [06:09:20] <pieter_hintjens> i’m wondering if it’s worth resurrecting the old amqp file transfer semantics as an example over 0MQ
[Sunday 05 September 2010] [06:09:26] <pieter_hintjens> should be nice and easy to implement
[Sunday 05 September 2010] [06:09:41] <sustrik> if you have resources to do that
[Sunday 05 September 2010] [06:09:58] <sustrik> an example?
[Sunday 05 September 2010] [06:10:00] <pieter_hintjens> well, it’s a question of what examples are the most valuable
[Sunday 05 September 2010] [06:10:24] <pieter_hintjens> would you rather have a name resolution service or a file transfer service over 0MQ?
[Sunday 05 September 2010] [06:10:27] <sustrik> ah, in the guide?
[Sunday 05 September 2010] [06:10:29] <pieter_hintjens> yeah
[Sunday 05 September 2010] [06:10:43] <sustrik> afaiu the examples in the guide are to be simple
[Sunday 05 September 2010] [06:10:53] <sustrik> file transfer is rather complex
[Sunday 05 September 2010] [06:10:56] <pieter_hintjens> sustrik: with 0MQ everything is simple :-)
[Sunday 05 September 2010] [06:11:03] <sustrik> i mean chopping the files
[Sunday 05 September 2010] [06:11:09] <sustrik> reassemblig them
[Sunday 05 September 2010] [06:11:14] <sustrik> etc.
[Sunday 05 September 2010] [06:11:38] <pieter_hintjens> maybe
[Sunday 05 September 2010] [06:12:00] <pieter_hintjens> actually it’s not that complex, really
[Sunday 05 September 2010] [06:12:06] <sustrik> it’s up to you
[Sunday 05 September 2010] [06:12:17] <pieter_hintjens> the sophisticated part is the stage / delivery semantics but that’s a neat demo of 0MQ IMO
[Sunday 05 September 2010] [06:12:24] <pieter_hintjens> we’ll see...
[Sunday 05 September 2010] [06:12:33] <sustrik> optinally, it can be a tutorial
[Sunday 05 September 2010] [06:12:40] <pieter_hintjens> yes
[Sunday 05 September 2010] [06:12:52] <pieter_hintjens> it reminded me that someone said “80% of messaging is still file transfer”
[Sunday 05 September 2010] [06:13:33] <pieter_hintjens> and it kind of seems that by covering the extreme other side of messaging (from low-latency) we prove the generality of the model
[Sunday 05 September 2010] [06:14:01] <pieter_hintjens> also it fits into the notion of distributed device management via shuffling config files around
[Sunday 05 September 2010] [06:14:05] * pieter_hintjens will think about it
[Sunday 05 September 2010] [06:14:11] <sustrik> ok
[Sunday 05 September 2010] [06:14:39] <pieter_hintjens> we will need a simple tutorial on how to make and submit git patches
[Sunday 05 September 2010] [06:14:57] <sustrik> enumerating the steps won’t do?
[Sunday 05 September 2010] [06:15:07] <pieter_hintjens> that would be a simple tutorial, then
[Sunday 05 September 2010] [06:15:10] <pieter_hintjens> it would do, yes
[Sunday 05 September 2010] [06:15:35] <sustrik> gitt diff, send as a main, put [PATCH] in the subject, state it’s under MIT
[Sunday 05 September 2010] [06:15:43] <sustrik> that’s it, no?
[Sunday 05 September 2010] [06:15:59] <sustrik> git, mail*
[Sunday 05 September 2010] [06:16:12] <pieter_hintjens> roughly, but some of the details cause problems
[Sunday 05 September 2010] [06:16:32] <pieter_hintjens> “Honestly? I spent too much time being stuck. I ended up on some crappy tutorial that said to type “git-format-patch”, when of course it was “git format-patch”. I got stuck on that until I trial-and-error’d.”
[Sunday 05 September 2010] [06:16:58] <sustrik> so far we don’t require formatted patches
[Sunday 05 September 2010] [06:17:05] <sustrik> so simple diff would do
[Sunday 05 September 2010] [06:17:11] <sustrik> not even git diff is needed
[Sunday 05 September 2010] [06:17:22] <sustrik> but it would be nice if people sent formatted patches
[Sunday 05 September 2010] [06:17:22] <pieter_hintjens> mato was saying something about signed-off-by
[Sunday 05 September 2010] [06:17:31] <sustrik> yup
[Sunday 05 September 2010] [06:17:33] <pieter_hintjens> indeed
[Sunday 05 September 2010] [06:17:56] <pieter_hintjens> for later...
[Sunday 05 September 2010] [06:18:14] <sustrik> would be nice to document it though
[Sunday 05 September 2010] [06:18:35] <pieter_hintjens> yes
[Sunday 05 September 2010] [06:18:46] <pieter_hintjens> did you update the contract page or shall I do that
[Sunday 05 September 2010] [06:18:53] <pieter_hintjens> i thought the way you expressed it in the email was very clear
[Sunday 05 September 2010] [06:19:07] <pieter_hintjens> but it needs to be written down, or it’ll get lost
[Sunday 05 September 2010] [06:20:16] <sustrik> ok, i’ll do that
[Sunday 05 September 2010] [06:20:44] <pieter_hintjens> what we found, in FFII work, was that it was usually better to edit the page first, then post to the list explaining the change
[Sunday 05 September 2010] [06:21:03] <pieter_hintjens> this is kind of a standard pattern to avoid losing stuff in the email list
[Sunday 05 September 2010] [06:25:33] <sustrik> ok
[Sunday 05 September 2010] [06:25:43] <sustrik> btw, the inproc example
[Sunday 05 September 2010] [06:26:04] <sustrik> you can fix it by launch step1’s thread from step2
[Sunday 05 September 2010] [06:26:13] <sustrik> that should synchronise it
[Sunday 05 September 2010] [06:27:39] <pieter_hintjens> hmm, nice solution...
[Sunday 05 September 2010] [06:28:50] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [06:31:39] <pieter_hintjens> ack, that works
[Sunday 05 September 2010] [06:31:45] <sustrik> great
[Sunday 05 September 2010] [06:35:44] <guido_g> hi
[Sunday 05 September 2010] [06:35:58] <guido_g> just saw the patch thing
[Sunday 05 September 2010] [06:36:25] <guido_g> I used git format-patch
[Sunday 05 September 2010] [06:36:44] <guido_g> but i had my changes in a local branch
[Sunday 05 September 2010] [06:37:52] <sustrik> guido_g: hi
[Sunday 05 September 2010] [06:37:59] <sustrik> i am not an expert on git
[Sunday 05 September 2010] [06:38:12] <sustrik> do you need a local branch to be able to format a patch?
[Sunday 05 September 2010] [06:38:47] <guido_g> i’m no expert neither, but will have a look
[Sunday 05 September 2010] [06:39:20] <sustrik> looks like there’s a “single-commit” mode
[Sunday 05 September 2010] [06:42:52] <guido_g> git format-patch -k –stdout R1..R2 <- from the examples of git help format-patch
[Sunday 05 September 2010] [06:43:08] <guido_g> seems that it works for commits and ranges of commits
[Sunday 05 September 2010] [06:43:32] <sustrik> nice
[Sunday 05 September 2010] [06:43:38] <guido_g> git format-patch -3 <- sweet
[Sunday 05 September 2010] [06:43:54] <guido_g> last three commits as patches
[Sunday 05 September 2010] [06:44:22] <sustrik> bingo
[Sunday 05 September 2010] [06:44:28] <sustrik> that’s what most people need
[Sunday 05 September 2010] [06:44:36] <guido_g> uh
[Sunday 05 September 2010] [06:45:06] * guido_g thinks changes should be done in branches
[Sunday 05 September 2010] [06:46:11] <sustrik> they should, but you don’t expect newbies submitting patches to deal with branching
[Sunday 05 September 2010] [06:46:29] <sustrik> installing git and learning how to commit is hard enough
[Sunday 05 September 2010] [06:46:54] <sustrik> that’s why i am happy with applying simple git diffs for now
[Sunday 05 September 2010] [06:47:20] <sustrik> or even dumb old diffs
[Sunday 05 September 2010] [06:47:55] <guido_g> ok
[Sunday 05 September 2010] [06:48:07] <sustrik> step-by-step would be nice though
[Sunday 05 September 2010] [06:48:08] <guido_g> so I spend hours for nothing...
[Sunday 05 September 2010] [06:48:10] <guido_g> :)
[Sunday 05 September 2010] [06:48:18] <sustrik> i mean, that’s the goal
[Sunday 05 September 2010] [06:48:25] <sustrik> we have to educate people
[Sunday 05 September 2010] [06:48:29] <guido_g> right
[Sunday 05 September 2010] [06:48:47] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [06:48:48] <sustrik> do we have a patch process described anywhere?
[Sunday 05 September 2010] [06:48:57] <sustrik> hm
[Sunday 05 September 2010] [06:49:05] * guido_g runs for cover
[Sunday 05 September 2010] [06:49:59] <pieter_hintjens> gudio_g: i think branches are orthogonal to patches
[Sunday 05 September 2010] [06:50:11] <pieter_hintjens> in the simplest case, you clone a repo, make changes, do a diff or git diff, and get a patch
[Sunday 05 September 2010] [06:50:34] <pieter_hintjens> well, that’s the medium hard case
[Sunday 05 September 2010] [06:50:53] <pieter_hintjens> in the simplest case you take the source code package, make changes, do a diff and get a non-git patch
[Sunday 05 September 2010] [06:51:13] <pieter_hintjens> but that only applies to projects that make source code packages
[Sunday 05 September 2010] [06:51:20] <sustrik> ack, and the hard case requires a topc branch
[Sunday 05 September 2010] [06:51:26] <sustrik> topic*
[Sunday 05 September 2010] [06:52:27] <pieter_hintjens> yes, exactly
[Sunday 05 September 2010] [06:53:07] <pieter_hintjens> for projects like zguide, there is no source package
[Sunday 05 September 2010] [06:53:24] <pieter_hintjens> we have to educate people, as you say
[Sunday 05 September 2010] [06:54:05] <sustrik> we are at the beginning of the road, it’ll take some time to make this work
[Sunday 05 September 2010] [06:54:13] <sustrik> but it’s worth of doing
[Sunday 05 September 2010] [06:55:38] <sustrik> guido_g: can you write down your experience with format-patch?
[Sunday 05 September 2010] [06:56:35] <sustrik> it could be used as a basis for the patch process tutorial
[Sunday 05 September 2010] [06:57:54] <guido_g> hmmm
[Sunday 05 September 2010] [06:58:11] <guido_g> i did it by book, three steps and done
[Sunday 05 September 2010] [06:58:22] <guido_g> but if you really want it...
[Sunday 05 September 2010] [06:58:37] <sustrik> yes, please
[Sunday 05 September 2010] [06:58:43] <sustrik> you’ll spare me an hour :)
[Sunday 05 September 2010] [06:58:46] <pieter_hintjens> guido_g: yes please, we’ll have to document the whole thing
[Sunday 05 September 2010] [06:58:50] <guido_g> sigh
[Sunday 05 September 2010] [06:58:52] <pieter_hintjens> how to submit patch from source package
[Sunday 05 September 2010] [06:59:09] <pieter_hintjens> how to submit patch from git repository clone
[Sunday 05 September 2010] [06:59:12] <guido_g> ok, then i’ll write down what I did
[Sunday 05 September 2010] [06:59:17] <sustrik> thanks
[Sunday 05 September 2010] [06:59:18] <pieter_hintjens> how to make topic branches and then submit them as patches
[Sunday 05 September 2010] [06:59:37] <pieter_hintjens> can you create a page on the wiki, minimal, and we can take it from there?
[Sunday 05 September 2010] [06:59:52] <pieter_hintjens> you should be able to create pages in the docs: category
[Sunday 05 September 2010] [07:00:06] <pieter_hintjens> “we” = everyone who cares about this
[Sunday 05 September 2010] [07:01:02] <guido_g> this cross-domain cookie thing sucks
[Sunday 05 September 2010] [07:02:38] <sustrik> ah, yes
[Sunday 05 September 2010] [07:02:53] <sustrik> pieter, any idea what’s the problem?
[Sunday 05 September 2010] [07:02:56] <guido_g> ok, where’s the how-to for the wiki?
[Sunday 05 September 2010] [07:03:09] <guido_g> sustrik: it’s a wikidot thing
[Sunday 05 September 2010] [07:03:12] <sustrik> guido_g: just post the command lines here
[Sunday 05 September 2010] [07:03:18] <guido_g> ok
[Sunday 05 September 2010] [07:11:33] <guido_g> that’s the whole magic
[Sunday 05 September 2010] [07:12:26] <guido_g> “git format-patch -1” is really nice
[Sunday 05 September 2010] [07:13:22] <sustrik> thanks!
[Sunday 05 September 2010] [07:16:40] <jond> hi
[Sunday 05 September 2010] [07:16:47] <sustrik> jon: hi
[Sunday 05 September 2010] [07:16:59] <jond> good to see a policy round the patches
[Sunday 05 September 2010] [07:17:11] <sustrik> ack
[Sunday 05 September 2010] [07:17:12] <sustrik> will you be able to make it for the meetup on tuesday?
[Sunday 05 September 2010] [07:17:35] <jond> I’ve had trouble squahsing commits in the past, which are meant to be easier for maintainer
[Sunday 05 September 2010] [07:17:44] <sustrik> i would like to have a talk with you about the subscription forwarding
[Sunday 05 September 2010] [07:18:07] <jond> tues: should be ok, though had to re-arrange a family thing
[Sunday 05 September 2010] [07:18:20] <sustrik> ah, sorry about that
[Sunday 05 September 2010] [07:19:16] <sustrik> discussing development via emails takes much more time than having a private talk
[Sunday 05 September 2010] [07:19:16] <jond> also; i can’t see irc in the dayjob, or twitter etc so if there’s any last minute change mailing list is best
[Sunday 05 September 2010] [07:19:41] <sustrik> jond: ok, will do
[Sunday 05 September 2010] [07:21:22] <jond> on the subject of git, the chacon book seems to recommend git pull requests and format patch. it’s all very powerful but seems easy to hang yrself
[Sunday 05 September 2010] [07:22:08] <sustrik> we’ll refine the process as we get better in using git
[Sunday 05 September 2010] [07:22:09] <guido_g> format-patch itself is quite nice, as shown above
[Sunday 05 September 2010] [07:22:35] <jond> am I meant to bring my local patch branches upto date with master/maint before running the diff/format-patch?
[Sunday 05 September 2010] [07:23:01] <sustrik> jond: that would be nice
[Sunday 05 September 2010] [07:23:31] <sustrik> otherwise it’s up to me to resolve the conflicts
[Sunday 05 September 2010] [07:23:40] <sustrik> which can in turn break your code
[Sunday 05 September 2010] [07:24:12] <sustrik> well, it depends
[Sunday 05 September 2010] [07:24:25] <sustrik> if you are submitting a single-line patch
[Sunday 05 September 2010] [07:24:34] <sustrik> and there have been a lot of change on master
[Sunday 05 September 2010] [07:24:46] <sustrik> it’s probably easier for me to merge it then for you
[Sunday 05 September 2010] [07:24:50] <jond> I also think we should recommend squashing the commits, as it makes the patch simpler to read (if git rebase -i works that is)
[Sunday 05 September 2010] [07:25:25] <sustrik> squash = rebase to the actual head?
[Sunday 05 September 2010] [07:25:55] <guido_g> already going way to far
[Sunday 05 September 2010] [07:26:09] <jond> i am trying to keep in line with 2.0.9 and maint at the moment. what mato seems to have put in place is similar to waht we do with clearcase in the day job.
[Sunday 05 September 2010] [07:26:41] <guido_g> pull -> [branch or not] -> changes -> commit [may be repeated] -> format-patch -> mailing list
[Sunday 05 September 2010] [07:27:59] <jond> well in the past, I used the rebase to make all the individual commits I did on my branch appear as 1 for delivery. It’s really annoying to have seperate commits for typo’s etc
[Sunday 05 September 2010] [07:28:41] <guido_g> but it’s great if you have different changes made in a row
[Sunday 05 September 2010] [07:28:52] <guido_g> so the maintainer can the ones he likes
[Sunday 05 September 2010] [07:29:25] <guido_g> anyhow, i think i’ll stick with this easy way
[Sunday 05 September 2010] [07:29:55] <jond> i’d use a seperate branch for each change.
[Sunday 05 September 2010] [07:30:33] <guido_g> if you do large changes, that might be ok
[Sunday 05 September 2010] [07:30:53] <guido_g> for smaller ones it’s sounds like a nightmare
[Sunday 05 September 2010] [07:32:32] <sustrik> jond: i wouldn’t bother with maint branch for the development work
[Sunday 05 September 2010] [07:32:54] <sustrik> it’s meant rather for small fixes
[Sunday 05 September 2010] [07:34:04] <sustrik> guido_g: yes, the smaller the patch the simpler the process should be
[Sunday 05 September 2010] [07:34:26] <sustrik> that’s why i’m saying i would accept even plain old diffs
[Sunday 05 September 2010] [07:34:45] <guido_g> great
[Sunday 05 September 2010] [07:35:02] <guido_g> lowers the entry barrier
[Sunday 05 September 2010] [07:35:11] <sustrik> exactly
[Sunday 05 September 2010] [07:35:27] <sustrik> however, if you ar woking on new functionality that’ll take 1/2 yesr to complete
[Sunday 05 September 2010] [07:35:39] <sustrik> you definitely want your own branch
[Sunday 05 September 2010] [07:35:44] <guido_g> and i’d demand payment :)
[Sunday 05 September 2010] [07:35:51] <sustrik> good point :)
[Sunday 05 September 2010] [07:37:28] <guido_g> so, now that i burned my eggs for brunch while writing git examples, i’m in the right mood for writing some more tests...
[Sunday 05 September 2010] [07:37:38] <jond> the many branches thing pretty much also depends on the turnaround of the patch getting to master etc
[Sunday 05 September 2010] [07:38:40] <jond> guido_g : i should send you the prefix_tree test ( was on the mailing list, i think we need tests now)
[Sunday 05 September 2010] [07:39:19] <jond> sustrik: hopefully get chance to discuss that pub filtering on tuesday
[Sunday 05 September 2010] [07:41:12] <sustrik> jond: sorry, i’ve seen it, but i was busy since :(
[Sunday 05 September 2010] [07:42:10] Quit xla has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [07:43:41] <jond> yes me too; 0MQ stuff very intermittent , as dayjob is not involving it all.
[Sunday 05 September 2010] [07:44:13] <guido_g> sigh at least, I’m not alone...
[Sunday 05 September 2010] [07:44:23] <jond> incidently a SUB socket can bind can’t it and then mutiple PUB’s connect
[Sunday 05 September 2010] [07:46:35] <sustrik> jond: yes
[Sunday 05 September 2010] [07:47:13] <sustrik> thus the subscription should be forwarded to _all_ connected publishers
[Sunday 05 September 2010] [07:47:51] <jond> ok, then maybe that’s one for the docs, someone asked that last night on here, they wanted fan-in to single server, I don’t think we cover that in the docs.
[Sunday 05 September 2010] [07:48:39] <sustrik> it kind of follows from the orthogonality of socket types and binds/connects
[Sunday 05 September 2010] [07:48:46] Quit keffo has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [07:48:53] <sustrik> but yes, it would be nice to mention it somewhere
[Sunday 05 September 2010] [07:49:02] <sustrik> maybe in the guide?
[Sunday 05 September 2010] [07:49:40] <jond> sustrik: sorry I meant the guide
[Sunday 05 September 2010] [07:50:09] <sustrik> pieter_hintjens: see above, suggestion for the guide
[Sunday 05 September 2010] [07:51:19] <sustrik> btw, the fact that binds/connects are independent of socket types is something that most people don’t get immediately
[Sunday 05 September 2010] [07:55:22] <jond> yes, i think that is the problem. there’s that big book enterprise messaging or whatever be nice to lift some scenarios from there and put in guide in 0MQ terms, which is pretty much what ph has been doing very well.
[Sunday 05 September 2010] [07:56:23] <sustrik> yes, it seems to work
[Sunday 05 September 2010] [07:57:08] <sustrik> btw, i’ve already seen some job ads requiring 0MQ knowledge so hopefully we’ll get into the phase where people are actually paid for working on 0MQ
[Sunday 05 September 2010] [07:57:11] <sustrik> we’ll see
[Sunday 05 September 2010] [07:58:55] <jond> that’s interesting news. I can see it being difficult to get 0MQ in some places with established code (understandly so in some cases, like reliable multicast)
[Sunday 05 September 2010] [08:01:48] <sustrik> well, i don’t expect nasdaq to replace their infrastrucutre by 0mq any time soon :)
[Sunday 05 September 2010] [08:02:11] <sustrik> but small companies are more flexible in adopting new technologies
[Sunday 05 September 2010] [08:05:11] <guido_g> hopefully
[Sunday 05 September 2010] [08:05:37] <guido_g> would be nice to have a mq related day job
[Sunday 05 September 2010] [08:09:08] <sustrik> my guess is that the early adopters are stock trading start-ups
[Sunday 05 September 2010] [08:09:46] <sustrik> that would mean that places like NY, London or Chicago are most interesting
[Sunday 05 September 2010] [08:09:55] <sustrik> i may be wrong though
[Sunday 05 September 2010] [08:10:35] <guido_g> hmmm... with mongrel2 there might be even ordinary web-work available
[Sunday 05 September 2010] [08:10:51] <sustrik> interesting point
[Sunday 05 September 2010] [08:11:11] <sustrik> i haven’t thought of that as i never done anything with web myself
[Sunday 05 September 2010] [08:11:29] <guido_g> i left it in early 2005 and never looked back :)
[Sunday 05 September 2010] [08:11:41] <sustrik> :)
[Sunday 05 September 2010] [08:11:44] <guido_g> never did fronted things anyway
[Sunday 05 September 2010] [08:12:13] <lestrrat> I doubt mongrel2 would really generate that much 0mq jobs... not at least from the web developer’s standpoint.
[Sunday 05 September 2010] [08:12:37] <lestrrat> zeromq is completely abstracted away in the web framework layer, so the web developers won’t even know ;)
[Sunday 05 September 2010] [08:12:57] <sustrik> right – unless you want to scale
[Sunday 05 September 2010] [08:13:06] <sustrik> what if you want N web servers
[Sunday 05 September 2010] [08:13:07] <lestrrat> of course, some of the more curious guys will do interesting web stuff with 0mq
[Sunday 05 September 2010] [08:13:14] <sustrik> and load-balanced services
[Sunday 05 September 2010] [08:13:22] <sustrik> persistence in the middle, etc.
[Sunday 05 September 2010] [08:13:40] <sustrik> that are google-like scenarios afaics
[Sunday 05 September 2010] [08:14:04] <guido_g> would also be nice to have it for mor ebay like things
[Sunday 05 September 2010] [08:14:08] <guido_g>
*more
[Sunday 05 September 2010] [08:14:14] <lestrrat> well, you’re talking about the smart guys, they’ll do anything. your average joe web devs won’t touch it with a 40ft pole
[Sunday 05 September 2010] [08:14:16] <lestrrat> ;)
[Sunday 05 September 2010] [08:14:36] <guido_g> lestrrat: you’re wrong
[Sunday 05 September 2010] [08:14:42] <guido_g> they do touch it
[Sunday 05 September 2010] [08:14:48] <guido_g> which is the problem...
[Sunday 05 September 2010] [08:15:05] <lestrrat> I knew where that was going ;)
[Sunday 05 September 2010] [08:15:17] <sustrik> :)
[Sunday 05 September 2010] [08:15:38] <guido_g> hehe
[Sunday 05 September 2010] [08:24:37] Quit keffo has left this server (Ping timeout: 260 seconds).
[Sunday 05 September 2010] [08:31:23] <guido_g> q: where do i find the declaration of int64_t?
[Sunday 05 September 2010] [08:33:24] <sustrik> guido_g: stdint.h
[Sunday 05 September 2010] [08:33:36] <guido_g> thx
[Sunday 05 September 2010] [09:06:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [09:15:09] <jond> sustrik: yes, for the large orgs change more likely thru acquisition/merger so small is best for more leading stuff
[Sunday 05 September 2010] [09:16:08] <jond> re : mongrel2 does anyone understand how the 0MQ integration works? It uses that c based co-routine library?
[Sunday 05 September 2010] [09:18:08] Quit xla has left this server (Read error: Operation timed out).
[Sunday 05 September 2010] [09:37:22] <sustrik> jond: i think so, but i haven’t checked
[Sunday 05 September 2010] [11:02:38] Quit
xla_ has left this server (Ping timeout: 245 seconds).
[Sunday 05 September 2010] [11:12:42] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [11:40:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [11:54:32] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [11:57:20] Quit xla has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [12:27:54] Part andrewvc has left this channel.
[Sunday 05 September 2010] [12:31:10] Quit AndrewBC has left this server (Ping timeout: 272 seconds).
[Sunday 05 September 2010] [12:37:26] Quit keffo has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [13:25:56] <cremes> Tasser: i’m not on irc very much during the weekends (Chicago, IL, US); if you have questions i recommend you send them to the mailing list
[Sunday 05 September 2010] [13:37:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [14:05:27] <pieter_hintjens> sustrik: re, was out with the family in the woods...
[Sunday 05 September 2010] [14:08:03] <pieter_hintjens> i will cover publishers connecting to subscriber when i find a sensible use case
[Sunday 05 September 2010] [14:08:28] <pieter_hintjens> “this is what you can do” is not useful but “this is how you solve problem X” is useful
[Sunday 05 September 2010] [14:09:10] <pieter_hintjens> fan-in to single server... maybe for collecting statistics
[Sunday 05 September 2010] [14:09:18] <pieter_hintjens> though why not use PUSH rather than PUB...?
[Sunday 05 September 2010] [14:30:30] <cremes> pieter_hintjens: fyi, there are some comments on that igvita article that probably need a response
[Sunday 05 September 2010] [14:30:42] <pieter_hintjens> cremes: am actually doing that right now
[Sunday 05 September 2010] [14:30:57] <cremes> i would respond if i knew anything about boost::asio or sc-whatever
[Sunday 05 September 2010] [14:31:00] <cremes> cool
[Sunday 05 September 2010] [14:31:00] <pieter_hintjens> though I doubt they “need” a response except “download it and try it”
[Sunday 05 September 2010] [14:31:07] <cremes> ha
[Sunday 05 September 2010] [14:31:21] <pieter_hintjens> i’m reminded of the virgin declaring why sex is so pointless and messy
[Sunday 05 September 2010] [14:31:50] <cremes> it’s just a good idea to make sure comments like that don’t go unanswered especially when google turns up that article in 6 months
[Sunday 05 September 2010] [14:32:14] <pieter_hintjens> ack
[Sunday 05 September 2010] [14:32:26] <cremes> l8r
[Sunday 05 September 2010] [14:32:28] <pieter_hintjens> if you have specific questions you’d like me to pontificate on, send me the URLs
[Sunday 05 September 2010] [14:32:40] <cremes> ack :)
[Sunday 05 September 2010] [14:35:31] <sigmonsays> does zeroMQ implement AMQP?
[Sunday 05 September 2010] [14:35:38] <DasIch> sigmonsays: no
[Sunday 05 September 2010] [14:37:29] <sigmonsays> [newb] is there a doc that describes what protocols it uses? I mean i’m sure the same concepts exist that AMQP implements..
[Sunday 05 September 2010] [14:41:54] <pieter_hintjens> sigmonsays: rfc.zeromq.org but it’s out of date
[Sunday 05 September 2010] [14:42:09] <pieter_hintjens> this is one thing i have wanted to make for a while, a correct spec of the protocol
[Sunday 05 September 2010] [14:42:23] <pieter_hintjens> but it is very different to AMQP
[Sunday 05 September 2010] [14:42:56] <pieter_hintjens> AMQP has stupid clients talking to a smart broker
[Sunday 05 September 2010] [14:43:01] <pieter_hintjens> 0MQ has smart clients talking to each other
[Sunday 05 September 2010] [14:43:11] <pieter_hintjens> it means the wire level protocol can be a lot simpler
[Sunday 05 September 2010] [14:50:00] <zedas> jond: i do. it uses the coroutines and zmq_poll
[Sunday 05 September 2010] [14:52:38] <sigmonsays> So, if persistence were required, would that be something you’d implement on the sending side? How does the “broker” play a role if clients talk directly to each other? I’m having a hard time finding a doc that demonstrates a traditional setup
[Sunday 05 September 2010] [14:55:05] <sigmonsays> Thanks, will do
[Sunday 05 September 2010] [14:55:32] <sustrik> in short, there are so called “devices” which are basically “mini-brokers”; it you need a full fledged broker, there’s a project that allows you to plug rabbitmq into 0MQ network
[Sunday 05 September 2010] [14:56:32] <sigmonsays> Yah, I was scanning the amq C API to see what it implements. Quite sexy API =P
[Sunday 05 September 2010] [15:02:55] <sigmonsays> Is SCTP a supported transport?
[Sunday 05 September 2010] [15:03:18] <sustrik> it would be nice if it was, but no, it isn’t yet
[Sunday 05 September 2010] [15:04:39] <sustrik> actually, it used to be implemented in old versions of 0MQ but people weren’t much interested :|
[Sunday 05 September 2010] [15:05:20] <sustrik> it seems there’s still some kind of fear of using anything other than TCP
[Sunday 05 September 2010] [15:05:37] <pieter_hintjens> I proposed to the IETF that they use SCTP as the basis for WebSockets
[Sunday 05 September 2010] [15:05:42] <pieter_hintjens> the response was interesting
[Sunday 05 September 2010] [15:05:50] * sustrik is curious
[Sunday 05 September 2010] [15:06:06] <pieter_hintjens> something along the lines of “it’s not present on most boxes and therefore will never be used by anyone at all”
[Sunday 05 September 2010] [15:06:27] <sustrik> hm, right, it not on win boxes
[Sunday 05 September 2010] [15:06:31] <pieter_hintjens> yes indeed
[Sunday 05 September 2010] [15:06:43] <pieter_hintjens> end of story, pretty much
[Sunday 05 September 2010] [15:06:52] <Zao> Nor on most FreeBSD boxen, as it conflicts with things like VIMAGE, if you’re insane enough to want that.
[Sunday 05 September 2010] [15:07:38] <pieter_hintjens> i can’t login for some weird reason
[Sunday 05 September 2010] [15:09:14] <sustrik> hm, win64 comment
[Sunday 05 September 2010] [15:09:24] <sustrik> does anyone have a win64 installation here?
[Sunday 05 September 2010] [15:09:38] <sustrik> it should be easy to fix
[Sunday 05 September 2010] [15:09:45] <sustrik> if it doesn’t work
[Sunday 05 September 2010] [15:10:25] <pieter_hintjens> sorry, my virtual box is 32-bit...
[Sunday 05 September 2010] [15:10:46] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [15:12:40] <jond> ph: the fan-in question was asked by twcc on irc about 20:00-21:00 last night. they wanted multiple publisher to single server.
[Sunday 05 September 2010] [15:13:10] <pieter_hintjens> jond: do you recall the use case?
[Sunday 05 September 2010] [15:13:58] <pieter_hintjens> “anyone have a drill for me?”
[Sunday 05 September 2010] [15:14:01] <pieter_hintjens> “why?”
[Sunday 05 September 2010] [15:14:10] <pieter_hintjens> “i have a headache that won’t go away”
[Sunday 05 September 2010] [15:14:15] <jond> ph: shall i mail the irc, I didnt answer as I wasnt around; steve-o was
[Sunday 05 September 2010] [15:14:29] <Zao> sustrik: Does he refer to PGM being hard/impossible to build?
[Sunday 05 September 2010] [15:14:34] <jond> ph: you considering trepanation then?
[Sunday 05 September 2010] [15:14:40] <pieter_hintjens> well, exactly
[Sunday 05 September 2010] [15:14:54] <pieter_hintjens> before answering people’s questions about “how to do X”, ask them “why”...
[Sunday 05 September 2010] [15:14:57] <sustrik> zao: who are you referring to?
[Sunday 05 September 2010] [15:15:00] <pieter_hintjens> i can see the irc log, will check
[Sunday 05 September 2010] [15:15:32] <Zao> sustrik: I thought that by “win64” comment you referred to the lad talking about 64-bit windows in the reddit thread?
[Sunday 05 September 2010] [15:16:02] <pieter_hintjens> jond: bah, simple PUSH->PULL pattern would do it IMO
[Sunday 05 September 2010] [15:16:22] <sustrik> ah, yes, maybe he was referring to pgm rather than 0mq as such
[Sunday 05 September 2010] [15:16:32] <sustrik> i don’t have win32 box, cannot check
[Sunday 05 September 2010] [15:16:35] <sustrik> win64*
[Sunday 05 September 2010] [15:16:36] <jond> ph: IIRC correctly he wanted intermittent/transient clients sending no-ack-needed messages to single server. it was pretty vague
[Sunday 05 September 2010] [15:16:58] <Zao> sustrik: I’ve got a 64-bit Windows box, but I don’t trust mq on it at all.
[Sunday 05 September 2010] [15:17:10] <sustrik> Zao: why so?
[Sunday 05 September 2010] [15:17:13] <sustrik> does it fail?
[Sunday 05 September 2010] [15:17:40] <pieter_hintjens> jond: i just read it twice, he wants a push/pull fanin afaics
[Sunday 05 September 2010] [15:17:50] <pieter_hintjens> it’s covered in chapter 1 of the guide even
[Sunday 05 September 2010] [15:18:00] <pieter_hintjens> should be a standard answer to people who ask for help
[Sunday 05 September 2010] [15:18:14] <pieter_hintjens> “if you’ve not read ch1 & ch2 of the guide, do so and then come back”
[Sunday 05 September 2010] [15:18:15] <Zao> sustrik: I set up a PUB and bind. I set up another thread with a SUB and connect. I subscribe to all topics. I then send lots of short messages which the SUB gets, until after a few dozen, it just blocks in zmq::poll.
[Sunday 05 September 2010] [15:18:27] <Zao> sustrik: I’ve got to boil it down to a test case before I report it on the issue tracker.
[Sunday 05 September 2010] [15:18:36] <Zao> As well as testing it with another compiler.
[Sunday 05 September 2010] [15:18:50] <sustrik> Zao: great, thanks
[Sunday 05 September 2010] [15:19:00] <Zao> This is 32-bit VC10, by the way.
[Sunday 05 September 2010] [15:19:30] <sustrik> ack
[Sunday 05 September 2010] [15:21:18] <jond> ph: yes, but you can also do a sub that binds and a pub that connects. There’s more than one way at present, which is good.
[Sunday 05 September 2010] [15:21:59] <sustrik> ph: centralised logging scanrio?
[Sunday 05 September 2010] [15:22:56] Quit andrewvc has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [15:25:54] <sigmonsays> Is the concept of headers up to the user to implement or does something like “routing-key” exist? ideally I wouldn’t want to subscribe based on the serialized data, but the headers accompanying it.
[Sunday 05 September 2010] [15:26:38] <sigmonsays> ie, ZMQ_SUBSCRIBE example from the user guide
[Sunday 05 September 2010] [15:27:28] <Zao> As far as I understand it, the topic is the prefix part of the data.
[Sunday 05 September 2010] [15:28:21] <sigmonsays> Well I wouldn’t want to unpack a serialized message to inspect it is where i’m getting
[Sunday 05 September 2010] [15:28:42] <sigmonsays> I suspect I would prefix the compress data with HTTP like headers, but i thought i’m missing something..
[Sunday 05 September 2010] [15:29:16] <sustrik> you can place your topic into one message part, data to another
[Sunday 05 September 2010] [15:29:34] <sustrik> the matching is always done on the first message part
[Sunday 05 September 2010] [15:29:44] <sigmonsays> that’s what I was missing. Thanks
[Sunday 05 September 2010] [15:31:23] <jond> sustrik: i think logging would be scenario for that use of pub/sub
[Sunday 05 September 2010] [15:32:06] <sustrik> jond: that’s what ph asked for, no?
[Sunday 05 September 2010] [15:33:58] <sigmonsays> Is there anything to facilitate message persistence? I am thinking of a WAN setup where aggregation points exist in each pop. Is that where a queue would fit in?
[Sunday 05 September 2010] [15:34:43] <DasIch> fyi logbook (a python library for logging) does now provide support for a zeromq handler/subscriber
[Sunday 05 September 2010] [15:35:08] <sustrik> you want messages just to be offloaded to disk, or do you want to survive crash/shutdown?
[Sunday 05 September 2010] [15:36:41] <sigmonsays> sustrik, good question, You mean HA replication versus disk persistence? I just want the message to be delivered and not lost/corrupted/etc.
[Sunday 05 September 2010] [15:37:01] <sustrik> i mean swapping vs. storage
[Sunday 05 September 2010] [15:37:56] <sigmonsays> Well swap dies when the box does. So that’s not ideal
[Sunday 05 September 2010] [15:38:40] <sustrik> then you have to put something that persists messages into the network
[Sunday 05 September 2010] [15:38:42] <jond> sustrik: i think so. you could probably do a really nice demo with tokyo tyrant which is the reverse index lookup thing of tokyo cabinet. especially if log messages are text
[Sunday 05 September 2010] [15:39:12] <sustrik> i’ve laready meantioned that there’s a project that allows you to use RabbitMQ broker as a node in 0mq network
[Sunday 05 September 2010] [15:39:14] <pieter_hintjens> jond: sorry, was writing a post on reddit
[Sunday 05 September 2010] [15:39:32] <pieter_hintjens> sustrik: centralized logging... yes, perhaps
[Sunday 05 September 2010] [15:39:48] <pieter_hintjens> why is that not a push/pull then?
[Sunday 05 September 2010] [15:40:12] <pieter_hintjens> i mean, pubsub is a natural pattern with subscribers that select data
[Sunday 05 September 2010] [15:40:19] <pieter_hintjens> centralized logging is a sink pattern
[Sunday 05 September 2010] [15:40:36] <pieter_hintjens> i don’t like using knives as screwdrivers, though it works
[Sunday 05 September 2010] [15:40:46] <sustrik> think of the case you would add another application listening to log feed
[Sunday 05 September 2010] [15:40:47] <sigmonsays> sustrik, I see, rabbitmq broker seems like overkill for persistence. I could see using tyrant/cabinet – Was wondering if any facilities were already provided. Still newb here.
[Sunday 05 September 2010] [15:40:55] <sustrik> would you want the logs to be load balanced?
[Sunday 05 September 2010] [15:41:04] <sustrik> i don’t think so
[Sunday 05 September 2010] [15:41:06] <pieter_hintjens> sustrik: then every single app needs to connect to TWO sinks
[Sunday 05 September 2010] [15:41:08] <pieter_hintjens> that’s insane
[Sunday 05 September 2010] [15:41:22] <pieter_hintjens> or you use a broker/forwarder
[Sunday 05 September 2010] [15:41:36] <sustrik> but not streamer
[Sunday 05 September 2010] [15:41:37] <pieter_hintjens> but then you don’t connect pubs to subs
[Sunday 05 September 2010] [15:41:42] <pieter_hintjens> no, not streamer
[Sunday 05 September 2010] [15:41:46] <pieter_hintjens> broker, in that case
[Sunday 05 September 2010] [15:41:47] <sustrik> => it’s pub/sub
[Sunday 05 September 2010] [15:42:03] <pieter_hintjens> why not push=>pull?
[Sunday 05 September 2010] [15:42:15] <sustrik> because push/pull load balances
[Sunday 05 September 2010] [15:42:19] <sustrik> rather than distributes
[Sunday 05 September 2010] [15:42:22] <pieter_hintjens> yeah, if you want multiple sinks
[Sunday 05 September 2010] [15:42:30] <sustrik> right
[Sunday 05 September 2010] [15:42:31] <pieter_hintjens> you’d do a push-pull to the broker and then pub-sub to all sinks
[Sunday 05 September 2010] [15:42:38] Quit gebi has left this server (Ping timeout: 264 seconds).
[Sunday 05 September 2010] [15:42:44] <pieter_hintjens> still not a use case for pub->sub connections
[Sunday 05 September 2010] [15:42:49] <sustrik> you can do that, but why?
[Sunday 05 September 2010] [15:43:00] <pieter_hintjens> because it’s a sink
[Sunday 05 September 2010] [15:43:03] <pieter_hintjens> not a subscriber
[Sunday 05 September 2010] [15:43:16] <pieter_hintjens> ... message queuing semantics work better
[Sunday 05 September 2010] [15:43:26] <pieter_hintjens> i don’t want to lose log messages if broker isn’t there
[Sunday 05 September 2010] [15:43:33] <pieter_hintjens> i don’t want subscriptions
[Sunday 05 September 2010] [15:43:42] <pieter_hintjens> i want everything from all my upstream nodes
[Sunday 05 September 2010] [15:43:44] <pieter_hintjens> it’s a sink
[Sunday 05 September 2010] [15:43:52] <pieter_hintjens> pubsub is just not accurate...
[Sunday 05 September 2010] [15:44:07] <pieter_hintjens> we do IMO need a new socket type
[Sunday 05 September 2010] [15:44:12] <pieter_hintjens> which is the bus
[Sunday 05 September 2010] [15:44:19] <sustrik> shrug, it depends on you, if you believe that pipelining is the right pattern for message logs go for it
[Sunday 05 September 2010] [15:44:22] <pieter_hintjens> ... as we discussed before wrt error messages internally
[Sunday 05 September 2010] [15:44:38] <pieter_hintjens> it’s not the right pattern, it’s part of the pattern
[Sunday 05 September 2010] [15:44:43] <sustrik> pub/sub is a bus
[Sunday 05 September 2010] [15:44:48] <jond> ph+sustrik: what if we said ‘audit’ rather than logging. centralised audit of all messages?
[Sunday 05 September 2010] [15:45:11] <pieter_hintjens> audit, yes, nothing can be lost
[Sunday 05 September 2010] [15:45:29] <pieter_hintjens> pubsub is a stream with no guaranteed start
[Sunday 05 September 2010] [15:45:38] <sustrik> then yes, you want something reliable, like req/rep or pipeline
[Sunday 05 September 2010] [15:45:43] Part RunningMike has left this channel.
[Sunday 05 September 2010] [15:45:54] <sustrik> that’s what most logs are
[Sunday 05 September 2010] [15:45:56] <pieter_hintjens> yes
[Sunday 05 September 2010] [15:45:58] <sustrik> audit is different
[Sunday 05 September 2010] [15:46:04] <pieter_hintjens> logs are audits
[Sunday 05 September 2010] [15:46:12] <Zao> Oh joy. Even more fun behaviour sneaks up. zmq::poll(NULL, 0, t) asserts as winsock select craps itself on maxfd=~0.
[Sunday 05 September 2010] [15:46:14] <pieter_hintjens> losing a single message can be fatal
[Sunday 05 September 2010] [15:46:31] <pieter_hintjens> Zao: FD_SETSIZE?
[Sunday 05 September 2010] [15:47:01] Quit mato has left this server (Ping timeout: 260 seconds).
[Sunday 05 September 2010] [15:47:03] <pieter_hintjens> what is ~0...?
[Sunday 05 September 2010] [15:47:07] <sustrik> nope, zero-sized pollset
[Sunday 05 September 2010] [15:47:18] Quit keffo has left this server (Ping timeout: 252 seconds).
[Sunday 05 September 2010] [15:47:23] Quit andrewvc has left this server (Ping timeout: 265 seconds).
[Sunday 05 September 2010] [15:47:24] <sustrik> win32 select obviously don’t handle it well
[Sunday 05 September 2010] [15:47:36] * pieter_hintjens vaguely remembers something about that
[Sunday 05 September 2010] [15:47:38] <Zao> pieter_hintjens: maxfd is initialized to zmq::retired_fd and never gets assigned to anything else, as itemcount is 0.
[Sunday 05 September 2010] [15:47:58] <sustrik> Zao: can you report it please?
[Sunday 05 September 2010] [15:47:59] <Zao> winsock whines about invalid parameter, and the only one looking shady is the maxfd.
[Sunday 05 September 2010] [15:48:06] <Zao> sustrik: Yeah.
[Sunday 05 September 2010] [15:48:10] <sustrik> thanks
[Sunday 05 September 2010] [15:51:58] <pieter_hintjens> it looks to me that logging is a good usecase for a broker in fact
[Sunday 05 September 2010] [15:52:09] <pieter_hintjens> i’ll add this to the list of examples for the guide
[Sunday 05 September 2010] [15:55:39] <pieter_hintjens> sustrik: would it be fair to generalize that any N-to-N architecture needs a device in the middle?
[Sunday 05 September 2010] [15:55:49] <sustrik> definitely
[Sunday 05 September 2010] [15:55:53] <pieter_hintjens> except multicast, perhaps
[Sunday 05 September 2010] [15:55:55] <pieter_hintjens> ok
[Sunday 05 September 2010] [15:56:09] <sustrik> even with multicast you need a network switch in the middle
[Sunday 05 September 2010] [15:56:15] <pieter_hintjens> yes!
[Sunday 05 September 2010] [15:56:21] <sustrik> which is actaully a HW forwarder device
[Sunday 05 September 2010] [15:56:25] <pieter_hintjens> this is worth stating explicitly, it’s not clear initially why we need devices
[Sunday 05 September 2010] [15:56:32] <pieter_hintjens> i shall do this
[Sunday 05 September 2010] [15:56:37] <sustrik> ok
[Sunday 05 September 2010] [15:57:39] <Zao> Interesting. My testcase code doesn’t break.
[Sunday 05 September 2010] [15:57:42] <Zao> That’s annoying :D
[Sunday 05 September 2010] [15:57:58] <sustrik> you mean the poll bug?
[Sunday 05 September 2010] [15:58:14] <sustrik> it’s obvious what’s going on there, no need for test case
[Sunday 05 September 2010] [15:58:19] <sustrik> just report it
[Sunday 05 September 2010] [16:00:39] <Zao> sustrik: No, my other bug where poll on a SUB socket blocks even though the PUB peer keeps spamming it.
[Sunday 05 September 2010] [16:00:51] <Zao> sustrik: The one I’m trying to drum up a test case for, which doesn’t depend on D3D11.
[Sunday 05 September 2010] [16:01:00] <sustrik> ah
[Sunday 05 September 2010] [16:02:16] <pieter_hintjens> sustrik: you realize we’re totally screwed?
[Sunday 05 September 2010] [16:02:33] <sustrik> why so?
[Sunday 05 September 2010] [16:02:48] <pieter_hintjens> lol :-)
[Sunday 05 September 2010] [16:02:59] <pieter_hintjens> 10pm on a Sunday night and this is the most fun place to be
[Sunday 05 September 2010] [16:03:06] <sustrik> good god
[Sunday 05 September 2010] [16:03:09] <pieter_hintjens> yeah
[Sunday 05 September 2010] [16:03:20] <pieter_hintjens> and 0MQ is still under the radar...
[Sunday 05 September 2010] [16:03:47] <sustrik> i’ve been at a party, but returned to where fun is
[Sunday 05 September 2010] [16:03:58] <pieter_hintjens> splutter...!
[Sunday 05 September 2010] [16:04:02] <sustrik> :)
[Sunday 05 September 2010] [16:04:59] <pieter_hintjens> btw i added two concepts in the latest guide
[Sunday 05 September 2010] [16:05:17] <pieter_hintjens> 1. “0MQ string” -> length-delimited string with no trailing null
[Sunday 05 September 2010] [16:05:34] <pieter_hintjens> all the examples now correctly read/write strings in that format when they use strings
[Sunday 05 September 2010] [16:06:40] <sustrik> ok
[Sunday 05 September 2010] [16:08:38] <Zao> Hrm. Docs say that it should completely ignore the maxfd parameter.
[Sunday 05 September 2010] [16:11:54] Join snerd has joined this channel (
~motk@fedora/robk).
[Sunday 05 September 2010] [16:12:11] <sustrik> Zao: i think it does
[Sunday 05 September 2010] [16:12:25] <sustrik> Win32 fd_set is not a real fd_set
[Sunday 05 September 2010] [16:12:37] <sustrik> instead it’s a vector of SOCKETs
[Sunday 05 September 2010] [16:12:56] <sustrik> it’s vector of size 0 which causes trouble imo
[Sunday 05 September 2010] [16:16:27] Quit pieter_hintjens has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:16:27] Quit
snerd_ has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:16:27] Quit ivan has left this server (
*.net
*.split).
[Sunday 05 September 2010] [16:18:53] <sigmonsays> How does using (e)pgm allows multicast to be used reliably?
[Sunday 05 September 2010] [16:19:10] <sustrik> check RFC3208
[Sunday 05 September 2010] [16:19:18] <sustrik> there’s a resend mechanism
[Sunday 05 September 2010] [16:19:57] <sigmonsays> rgr
[Sunday 05 September 2010] [16:22:39] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 05 September 2010] [16:38:14] Part pieter_hintjens has left this channel.
[Sunday 05 September 2010] [18:11:18] Quit jsimmons has left this server (Client Quit).
[Sunday 05 September 2010] [18:48:26] Quit DasIch has left this server (Read error: Operation timed out).
[Sunday 05 September 2010] [18:49:41] Quit keffo has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [19:32:05] Quit andrewvc has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [19:38:27] Part andrewvc has left this channel.
[Sunday 05 September 2010] [19:48:22] Quit keffo has left this server (Ping timeout: 255 seconds).
[Sunday 05 September 2010] [20:36:56] Quit DasIch has left this server (Ping timeout: 276 seconds).
[Sunday 05 September 2010] [22:18:03] <mikejs> there’s no INSTALL file in the 2.0.9 tarball, despite it being referenced in the README
[Sunday 05 September 2010] [23:14:49] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 05 September 2010] [23:18:35] Quit iFire has left this server (Remote host closed the connection).
[Sunday 05 September 2010] [23:52:10] Quit sustrik has left this server (Quit: Leaving).
[Monday 06 September 2010] [01:10:36] <mato> mikejs: good catch, my fault for not testing one setting properly, i’ll fix the tarballs
[Monday 06 September 2010] [01:14:31] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [01:59:20] Quit AndrewBC has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [02:04:43] Quit bobs has left this server (Client Quit).
[Monday 06 September 2010] [02:11:43] Part bobs has left this channel.
[Monday 06 September 2010] [02:15:30] Part bobs has left this channel.
[Monday 06 September 2010] [02:35:04] Part bobs has left this channel.
[Monday 06 September 2010] [03:18:19] Part bobs has left this channel.
[Monday 06 September 2010] [03:55:50] Quit mcxx has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [04:16:08] <BooTheHamster> hi
[Monday 06 September 2010] [04:17:20] <mato> hi BooTheHamster
[Monday 06 September 2010] [04:17:26] <BooTheHamster> can I connect ZMQ socket to zero port number, e.g. “
tcp://localhost:0”?
[Monday 06 September 2010] [04:17:55] <mato> 0 is not a legal port number, so no :-)
[Monday 06 September 2010] [04:18:34] <BooTheHamster> ZMQ don’t get a random port no?
[Monday 06 September 2010] [04:18:47] <mato> ah, you want dynamically assigned ports on socket?
[Monday 06 September 2010] [04:18:52] <BooTheHamster> yes
[Monday 06 September 2010] [04:18:54] <mato> that’s a different question :)
[Monday 06 September 2010] [04:19:08] <BooTheHamster> sorry for my english :(
[Monday 06 September 2010] [04:19:12] <mato> no problem
[Monday 06 September 2010] [04:19:33] <mato> for dynamically assigned ports, that’s not possible at the moment but you can emulate it
[Monday 06 September 2010] [04:19:50] <mato> on the bind side, you’ll want to do a loop and try to bind to a random port in a portrange
[Monday 06 September 2010] [04:20:49] <mato> lines 588 onwards, “bind_to_random_port()”
[Monday 06 September 2010] [04:21:02] <mato> you’ll then want to save the port number you bound to, and connect to that
[Monday 06 September 2010] [04:21:16] <mato> it’s a hack, I hope to resolve this properly in 2.1.x
[Monday 06 September 2010] [04:21:21] <mato> but that is still some way away
[Monday 06 September 2010] [04:21:55] <BooTheHamster> great thanx, mato :)
[Monday 06 September 2010] [04:22:12] <mato> BooTheHamster: you’re welcome
[Monday 06 September 2010] [04:23:22] Quit BooTheHamster has left this server (Quit: Page closed).
[Monday 06 September 2010] [04:31:06] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [04:32:56] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [05:15:35] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [05:18:33] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 06 September 2010] [05:20:08] Quit rbraley has left this server (Ping timeout: 245 seconds).
[Monday 06 September 2010] [05:23:41] <Tasser> no tcl binding for 2.0 yet?
[Monday 06 September 2010] [06:17:38] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [07:44:01] <BooTheHamster> Hi. Can I use telnet (or something else) to debugging my application with ZMQ_REP socket?
[Monday 06 September 2010] [07:44:28] <keffo> I suggest a debugger.
[Monday 06 September 2010] [07:47:51] <BooTheHamster> I mean that I develop a server application, but don’t have a client yet. Can I sent a messages to opened ZMQ socket?
[Monday 06 September 2010] [07:48:21] <guido_g> write a simple client first then
[Monday 06 September 2010] [07:50:43] <BooTheHamster> thanx
[Monday 06 September 2010] [07:51:40] <guido_g> there is even simpler stuff in the tests directory on master
[Monday 06 September 2010] [07:52:48] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Monday 06 September 2010] [07:52:56] <BooTheHamster> thanx, guido_g.
[Monday 06 September 2010] [08:14:03] Part BooTheHamster has left this channel.
[Monday 06 September 2010] [09:04:47] Quit guido_g has left this server (Quit: Boom...).
[Monday 06 September 2010] [09:28:07] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [09:42:24] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [09:44:03] Quit iFire has left this server (Changing host).
[Monday 06 September 2010] [10:08:03] Nick
DasIch_ is now known as DasIch.
[Monday 06 September 2010] [11:28:02] <pieterh> “This sounds like just a big marketing gimmic,”
[Monday 06 September 2010] [11:28:52] <guido_g> reddit? yes...
[Monday 06 September 2010] [11:28:59] <guido_g> ]:->
[Monday 06 September 2010] [11:31:14] Quit plq has left this server (Ping timeout: 264 seconds).
[Monday 06 September 2010] [11:32:01] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Monday 06 September 2010] [12:27:38] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Monday 06 September 2010] [12:30:26] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 06 September 2010] [13:06:45] Quit plq has left this server (Ping timeout: 258 seconds).
[Monday 06 September 2010] [13:19:01] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [13:19:41] Quit xla has left this server (Ping timeout: 260 seconds).
[Monday 06 September 2010] [13:31:40] <sbahra> pieterh, in all honesty, I first heard about ZeroMQ from the multithreading magic article (which states things like “To scale further, or developer must switch to 100% lock-free algorithms for data sharing. He is now into the realm of black magic.”), and it made it seem like ZeroMQ was a marketing gimmick. :-P
[Monday 06 September 2010] [13:32:13] <sbahra> Now, we’re seriously looking at using ZeroMQ for the product I’m involved with. :-P
[Monday 06 September 2010] [13:32:17] <pieterh> sbahra :-)
[Monday 06 September 2010] [13:32:57] <pieterh> can you share what that product is?
[Monday 06 September 2010] [13:33:26] <sbahra> If we use it, I’ll let you know.
[Monday 06 September 2010] [13:33:34] <sbahra> pieterh, it’s a distributed high performance MTA.
[Monday 06 September 2010] [13:33:45] <pieterh> nice!
[Monday 06 September 2010] [13:34:02] <pieterh> i’ve thought for a while that 0MQ was ideal for clustered products
[Monday 06 September 2010] [13:34:14] <sbahra> Based on my basic readings so far, it seems like it.
[Monday 06 September 2010] [13:34:21] <sbahra> We’re currently using spread.
[Monday 06 September 2010] [13:34:31] * pieterh does not know spread
[Monday 06 September 2010] [13:34:40] <sbahra> Much older than 0MQ, I think. :-)
[Monday 06 September 2010] [13:35:12] <pieterh> yeah, found that via google...
[Monday 06 September 2010] [13:35:30] <pieterh> interesting...
[Monday 06 September 2010] [13:36:16] <pieterh> biggest difference I see so far is that Spread uses blue while 0MQ uses red
[Monday 06 September 2010] [13:37:05] <sbahra> “Spread currently supports application messages upto around 100Kbytes. Currently the exact size is not exported as part of the API.”
[Monday 06 September 2010] [13:37:15] <sbahra> pieterh, it doesn’t seem like that’s the case for 0MQ. :-P
[Monday 06 September 2010] [13:37:28] <pieterh> oh... their latency is measured in msec...
[Monday 06 September 2010] [13:37:31] <sbahra> pieterh, spread is very resilient.
[Monday 06 September 2010] [13:38:16] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [13:38:51] <pieterh> hmm, it’s interesting...
[Monday 06 September 2010] [13:38:53] <sbahra> Anyways, we’ll see if we switch over to 0MQ sometime next year.
[Monday 06 September 2010] [13:39:03] <sbahra> I should do some hackery dackery doe. Bye.
[Monday 06 September 2010] [13:39:05] <pieterh> covers a lot of the ground people complain about missing in 0MQ
[Monday 06 September 2010] [13:39:10] <pieterh> cyat
[Monday 06 September 2010] [13:58:18] Join icy has joined this channel (
~icy@lighttpd/icy).
[Monday 06 September 2010] [14:06:33] Quit pieterh has left this server (Ping timeout: 258 seconds).
[Monday 06 September 2010] [14:09:09] Nick
AndrewBC_ is now known as AndrewBC.
[Monday 06 September 2010] [14:18:48] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:21:27] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:29:32] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:31:40] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 06 September 2010] [14:41:19] Quit vlao has left this server (Read error: Connection reset by peer).
[Monday 06 September 2010] [14:48:21] Quit pieterh has left this server (Ping timeout: 260 seconds).
[Monday 06 September 2010] [15:48:55] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Monday 06 September 2010] [16:30:15] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 06 September 2010] [16:40:48] Quit andrewvc has left this server (Client Quit).
[Monday 06 September 2010] [16:41:49] Quit gebi has left this server (Ping timeout: 240 seconds).
*** on Mon Sep 6 16:59:51 2010
[Monday 06 September 2010] [16:59:51] Topic The channel topic is “Welcome!”.
[Monday 06 September 2010] [16:59:57] Mode Channel modes: no messages from outside, topic protection
[Monday 06 September 2010] [16:59:57] Created This channel was created on 2010-02-05 10:44.
[Monday 06 September 2010] [17:46:04] Part bobs has left this channel.
[Monday 06 September 2010] [17:47:41] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 06 September 2010] [19:03:43] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 06 September 2010] [20:36:59] Quit DasIch has left this server (Ping timeout: 272 seconds).
[Monday 06 September 2010] [21:39:53] Quit plq has left this server (Read error: Operation timed out).
[Monday 06 September 2010] [23:23:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [00:24:09] Quit keffo has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [01:27:02] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [01:48:19] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 07 September 2010] [01:54:18] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 07 September 2010] [01:54:29] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [01:57:45] <lestrrat> I’m having problems using fork() and making my parent process talk to the child processes via 0mq. I’m getting segfaults after the child exits and while the parent is in recv()
[Tuesday 07 September 2010] [01:57:50] <lestrrat> is this supposed to work?
[Tuesday 07 September 2010] [02:55:23] <pieterh> sustrik: you there?
[Tuesday 07 September 2010] [03:55:17] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Tuesday 07 September 2010] [04:18:48] Quit soren has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [04:32:49] Quit gebi has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [04:33:49] Quit xrfang has left this server (Remote host closed the connection).
[Tuesday 07 September 2010] [04:49:37] Quit plq has left this server (Read error: Operation timed out).
[Tuesday 07 September 2010] [05:07:21] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 07 September 2010] [05:20:45] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Tuesday 07 September 2010] [05:28:32] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 07 September 2010] [05:51:33] <pieterh> anyone here felt that XREP and XREQ could use better names?
[Tuesday 07 September 2010] [05:52:03] <lestrrat> I just gave a talk to my coworkers about zeromq, and yes, better_names++
[Tuesday 07 September 2010] [05:52:11] <pieterh> I was thinking of ROUTE and FORWARD
[Tuesday 07 September 2010] [05:53:24] * lestrrat doesn’t quite ring a bell
[Tuesday 07 September 2010] [05:53:58] <pieterh> XREP creates routing envelopes around incoming messages and uses these on output to route replies back to original clients
[Tuesday 07 September 2010] [05:54:09] Join plq has joined this channel (
~plq@78.189.30.218).
[Tuesday 07 September 2010] [05:54:12] <pieterh> XREQ just forwards messages in both directions without touching them
[Tuesday 07 September 2010] [05:55:08] <pieterh> XREP really looks like a router... it’s the only socket type that lets you address specific connections
[Tuesday 07 September 2010] [05:55:11] <keffo> I like ‘route’ very much, not forward so much
[Tuesday 07 September 2010] [05:55:36] <pieterh> yeah, forward wasn’t inspired
[Tuesday 07 September 2010] [05:55:51] <pieterh> it has to be a verb
[Tuesday 07 September 2010] [05:56:09] <pieterh> that says “move stuff in both directions but don’t mess with it”
[Tuesday 07 September 2010] [05:56:30] <pieterh> PORT
[Tuesday 07 September 2010] [05:56:45] <pieterh> XFER
[Tuesday 07 September 2010] [06:01:44] Quit andrewvc has left this server (Client Quit).
[Tuesday 07 September 2010] [06:11:56] <pieterh> lestrrat: does that ring a bell?
[Tuesday 07 September 2010] [06:13:11] <lestrrat> thinking
[Tuesday 07 September 2010] [06:15:38] <lestrrat> hmm. I grok ROUTE, but not the mudem part :) but I don’t have a great alternative plan either
[Tuesday 07 September 2010] [06:15:45] <lestrrat> naming is hard, eh.
[Tuesday 07 September 2010] [06:17:03] <pieterh> well, mudem is a play on modem, modulator/demodulator...
[Tuesday 07 September 2010] [06:17:09] Quit AndrewBC has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [06:17:14] <keffo> pieterh, I have a ‘route-codec’ in my code...
[Tuesday 07 September 2010] [06:17:18] <lestrrat> yeah, I know
[Tuesday 07 September 2010] [06:17:36] <pieterh> i don’t like invented words but if one has to invent them they should be expressive
[Tuesday 07 September 2010] [06:18:18] <keffo> whichi encodes/decodes routes for an xreq
[Tuesday 07 September 2010] [06:18:39] <pieterh> keffo: sounds right
[Tuesday 07 September 2010] [06:18:48] <pieterh> creates and uses envelopes, right?
[Tuesday 07 September 2010] [06:20:47] <keffo> it just handles them, contains a payload the ‘enduser’ is interested in, and also has a sendroute function
[Tuesday 07 September 2010] [06:20:48] <pieterh> well, an alternative to mudem: dispatch
[Tuesday 07 September 2010] [06:21:21] <keffo> naa, someone who implements any type of loadbalancing of messages is in essense a dispatcher, imo
[Tuesday 07 September 2010] [06:21:47] <pieterh> true
[Tuesday 07 September 2010] [06:22:06] <pieterh> it’s the combo of fanout and fanin
[Tuesday 07 September 2010] [06:23:06] <pieterh> i think “multiplex” is wrong since that suggest copying whereas its distribution
[Tuesday 07 September 2010] [06:23:40] <pieterh>
*it’s
[Tuesday 07 September 2010] [06:23:49] <lestrrat> yeah, I thought about multiplex, but it didn’t quite fit
[Tuesday 07 September 2010] [06:24:25] <pieterh> in terms of use cases, xreq is like push+pull, it ventilates and sinks at once
[Tuesday 07 September 2010] [06:25:24] <pieterh> one could create a nice pipeline pattern using just XREQ to XREQ
[Tuesday 07 September 2010] [06:27:11] <pieterh> how about... something more visual... 1TON
[Tuesday 07 September 2010] [06:27:57] <lestrrat> 1000kg!
[Tuesday 07 September 2010] [06:28:30] <pieterh> yeah
[Tuesday 07 September 2010] [06:28:41] <pieterh> 1-to-N for the pedantic of us
[Tuesday 07 September 2010] [06:31:05] <pieterh> it kind of feels more like a building block now
[Tuesday 07 September 2010] [06:42:46] <keffo> I have an issue where my worker process simply dissapears, but I cant seem to trap it
[Tuesday 07 September 2010] [06:42:57] <keffo> no exceptions, no atexits are run.. nada..
[Tuesday 07 September 2010] [06:42:58] <pieterh> what OS?
[Tuesday 07 September 2010] [06:42:59] <keffo> very annoying!
[Tuesday 07 September 2010] [06:43:01] <keffo> win7
[Tuesday 07 September 2010] [06:43:13] <pieterh> ah, that is a known problem
[Tuesday 07 September 2010] [06:43:21] <keffo> hu?
[Tuesday 07 September 2010] [06:43:23] <pieterh> the usual solution is to upgrade to Linux
[Tuesday 07 September 2010] [06:43:26] <pieterh> sorry :-)
[Tuesday 07 September 2010] [06:43:36] <keffo> caused by zmq??
[Tuesday 07 September 2010] [06:44:13] <keffo> tossing away 98% of the global userbase is hardly an upgrade btw :)
[Tuesday 07 September 2010] [06:44:14] <pieterh> I was kidding, my bad
[Tuesday 07 September 2010] [06:44:15] <pieterh> what language are you using?
[Tuesday 07 September 2010] [06:44:24] <keffo> c++, lua
[Tuesday 07 September 2010] [06:44:37] <pieterh> so you need a debug build of 0MQ IMO
[Tuesday 07 September 2010] [06:44:50] <keffo> oh it’s all debug, debugger is attached too :)
[Tuesday 07 September 2010] [06:44:58] <pieterh> aw :-(
[Tuesday 07 September 2010] [06:45:04] <keffo> gives me nothing.. I’ve tried all routes I can think of
[Tuesday 07 September 2010] [06:45:14] <keffo> abort()?
[Tuesday 07 September 2010] [06:45:20] <keffo> but why would that be called?
[Tuesday 07 September 2010] [06:45:20] <pieterh> DebugBreak() afair, then continue it in the debugger
[Tuesday 07 September 2010] [06:45:38] <pieterh> ... assertion failure?
[Tuesday 07 September 2010] [06:45:46] <keffo> nada :)
[Tuesday 07 September 2010] [06:46:12] <keffo> no asserts. no breakpoints, simply dissapears.. windows event log shows nothing
[Tuesday 07 September 2010] [06:46:16] <keffo> pussling actually
[Tuesday 07 September 2010] [06:46:21] <keffo> no exceptions are raised either
[Tuesday 07 September 2010] [06:46:40] <keffo> It’s as if the app cleanly exits, except it cant since it”s a while(true)
[Tuesday 07 September 2010] [06:46:57] <pieterh> it could exit in another thread I guess
[Tuesday 07 September 2010] [06:47:24] <pieterh> i’ve not worked on win32 for ages... maybe someone else here can be more helpful
[Tuesday 07 September 2010] [06:47:30] <keffo> does zmq ever call exit?
[Tuesday 07 September 2010] [06:47:39] <keffo> (or abort)
[Tuesday 07 September 2010] [06:47:41] * pieterh egreps the source code...
[Tuesday 07 September 2010] [06:47:48] Quit rbraley has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [06:48:04] <pieterh> nada
[Tuesday 07 September 2010] [06:48:09] <pieterh> asserts, yes
[Tuesday 07 September 2010] [06:48:43] <pieterh> 98%? keffo, 2010 is the Year of Linux
[Tuesday 07 September 2010] [06:48:51] <pieterh> it’s no more than 97.85% by now
[Tuesday 07 September 2010] [06:49:51] <keffo> if my mom could use any generic desktop linux without calling me, then I”d agree :)
[Tuesday 07 September 2010] [06:50:10] <pieterh> hah, my mum actually does use linux and has for years...
[Tuesday 07 September 2010] [06:51:02] <pieterh> but then again she’s currently asking me how to hide her IP address so she can troll Anonymous so perhaps she’s not typical...
[Tuesday 07 September 2010] [06:51:37] <pieterh> keffo: if you can make a reproducible case, and chop it down, maybe we can reproduce it on another platform
[Tuesday 07 September 2010] [06:52:18] <keffo> lord no, that would take ages :)
[Tuesday 07 September 2010] [06:52:34] <keffo> I just want to somehow detect -when- it happens, and go from there, but so far I’ve been unable to
[Tuesday 07 September 2010] [06:52:54] <pieterh> then, my friend, you might have to resort to...
[Tuesday 07 September 2010] [06:53:02] <pieterh> if really you have no other option...
[Tuesday 07 September 2010] [06:53:25] <keffo> print? =)
[Tuesday 07 September 2010] [06:53:28] <pieterh> yeah :-)
[Tuesday 07 September 2010] [06:53:38] <pieterh> don’t forget the fflush (stdout);
[Tuesday 07 September 2010] [06:54:52] <keffo> It’s remarkably reproducable though
[Tuesday 07 September 2010] [06:55:00] <pieterh> well, that’s always good
[Tuesday 07 September 2010] [06:55:03] <keffo> 4th time I calculate pi, it dissapears
[Tuesday 07 September 2010] [06:55:24] <pieterh> hopefully it remains stable as you add hundreds of prints
[Tuesday 07 September 2010] [06:55:40] <pieterh> you’re calculating pi?
[Tuesday 07 September 2010] [06:56:12] Quit mcxx has left this server (Quit: Lost terminal).
[Tuesday 07 September 2010] [06:57:24] <keffo> printing is not the problem, I generate ~250k of logs on each run :)
[Tuesday 07 September 2010] [06:57:42] <keffo> pi yeah, easy and verifiable thing to calc distributed :)
[Tuesday 07 September 2010] [06:58:04] <pieterh> is there an algo for distributed pi calculation somewhere?
[Tuesday 07 September 2010] [06:59:03] <keffo> sure
[Tuesday 07 September 2010] [06:59:08] <keffo> tons of different I guess
[Tuesday 07 September 2010] [06:59:10] <pieterh> i had an idea for a supermassive 0MQ project... lol
[Tuesday 07 September 2010] [06:59:39] <pieterh> not original but who cares...
[Tuesday 07 September 2010] [06:59:40] <keffo> as did I :)
[Tuesday 07 September 2010] [06:59:43] <keffo> for i=self.beginspan, self.endspan do
[Tuesday 07 September 2010] [06:59:43] <keffo> localpi = localpi + (1.0 / (i * 4.0 + 1.0) )
[Tuesday 07 September 2010] [06:59:43] <keffo> localpi = localpi - (1.0 / (i * 4.0 + 3.0) )
[Tuesday 07 September 2010] [07:00:15] <keffo> do that for each subspan of some arbitrary length, then sum them all up and the answer * 4 is pi :)
[Tuesday 07 September 2010] [07:00:30] <pieterh> not 42? weird...
[Tuesday 07 September 2010] [07:00:36] <keffo> hehe
[Tuesday 07 September 2010] [07:00:50] <pieterh> aight, so if we have a server somewhere that distributes workloads, and a simple 0MQ client that accepts them...
[Tuesday 07 September 2010] [07:01:52] <pieterh> has surely been done dozens of times
[Tuesday 07 September 2010] [07:02:08] <keffo> I’m doing a fairly more complicated scenario, but yeah
[Tuesday 07 September 2010] [07:05:18] <keffo> and I guess you can figure out why I want something a bit more flexibe than roundrobin :)
[Tuesday 07 September 2010] [07:06:00] <pieterh> right now I’m writing examples on how to use XREP to do routing
[Tuesday 07 September 2010] [07:06:25] <keffo> good, it was messy :)
[Tuesday 07 September 2010] [07:06:32] <pieterh> how so?
[Tuesday 07 September 2010] [07:06:40] <pieterh> you mean no documentation on the envelopes etc.?
[Tuesday 07 September 2010] [07:06:41] <keffo> just not very nicely explained
[Tuesday 07 September 2010] [07:06:47] <pieterh> right...
[Tuesday 07 September 2010] [07:11:26] Quit pieterh has left this server (Ping timeout: 264 seconds).
[Tuesday 07 September 2010] [07:13:09] <keffo> I would explain it as a stack+req...
[Tuesday 07 September 2010] [07:13:30] <keffo> push, push, push, payload, then pop,pop,pop, payload on the other side
[Tuesday 07 September 2010] [07:14:48] <keffo> btw, what happens in a queue device if a client never reconnects? will the msg linger indef.?
[Tuesday 07 September 2010] [07:21:02] <keffo> btw, what happens in a queue device if a client never reconnects? will the msg linger indef.?
[Tuesday 07 September 2010] [07:21:46] <pieterh> hmm, you mean a reply?
[Tuesday 07 September 2010] [07:21:53] <pieterh> with or without identity?
[Tuesday 07 September 2010] [07:22:19] <pieterh> this is what 0MQ/2.1 is fixing
[Tuesday 07 September 2010] [07:22:34] <pieterh> it will wait in some cases, discard in other cases
[Tuesday 07 September 2010] [07:22:41] <keffo> in general, the whole reconnect business
[Tuesday 07 September 2010] [07:22:53] <keffo> if it goes into the queue but never out, what happens to it?
[Tuesday 07 September 2010] [07:23:15] <pieterh> well, the queue is per socket, eventually
[Tuesday 07 September 2010] [07:23:52] * pieterh wishes the description of this had been documented beforehand
[Tuesday 07 September 2010] [07:24:13] <pieterh> there is not yet a proper explanation of how the 2.1 socket close semantics should work
[Tuesday 07 September 2010] [07:24:15] <pieterh> afaik
[Tuesday 07 September 2010] [07:24:29] <keffo> client(100msgs) -> queuedev -> service, then back again, except the client is gone forever..
[Tuesday 07 September 2010] [07:24:50] <pieterh> anonymous clients -> messages get thrown away
[Tuesday 07 September 2010] [07:25:06] <pieterh> client with identity -> messages persist as long as service is running
[Tuesday 07 September 2010] [07:25:25] <keffo> ok
[Tuesday 07 September 2010] [07:25:35] <pieterh> 0MQ does have the concept of a connection going away
[Tuesday 07 September 2010] [07:26:11] <pieterh> otherwise PUB sockets for example would end up with horrid resource leaks
[Tuesday 07 September 2010] [07:26:59] <keffo> I need to introduce some sort of session.. if I have (known)client A, does a bunch of test junk(like pi), but aborts prematurely, but then reconnects to start some other type of job, I dont want to receive a bunch of old pi results :)
[Tuesday 07 September 2010] [07:27:42] <keffo> and I would need to be able to tell all parties involved to dump everything related to an “old” session as well
[Tuesday 07 September 2010] [07:28:18] Quit mato has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [07:28:28] Quit plq has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [07:28:41] <pieterh> keffo: this starts to be industrial design work
[Tuesday 07 September 2010] [07:29:06] <keffo> pieterh, what do you mean?
[Tuesday 07 September 2010] [07:29:57] <pieterh> i mean, what you’re making is heavy duty...
[Tuesday 07 September 2010] [07:30:04] <keffo> oh very much :)
[Tuesday 07 September 2010] [07:30:25] <keffo> it has fried my brain on may occasions.. Tons of papers of diagrams spread all over the place :)
[Tuesday 07 September 2010] [07:30:32] <pieterh> if you have budget to throw at it, i can recommend an industrial 0MQ designer like Mato here
[Tuesday 07 September 2010] [07:31:07] <keffo> the bulk of the work is not the transport & topology though
[Tuesday 07 September 2010] [07:31:15] <keffo> although that needs to obviously be stable
[Tuesday 07 September 2010] [07:31:48] <pieterh> well, you need an infrastructure that understands ‘sessions’
[Tuesday 07 September 2010] [07:32:01] <keffo> Sure, but that”s already handled
[Tuesday 07 September 2010] [07:32:22] <pieterh> what do you still need then?
[Tuesday 07 September 2010] [07:32:32] <pieterh> apart from the thing not crashing...
[Tuesday 07 September 2010] [07:32:37] <keffo> hehe
[Tuesday 07 September 2010] [07:32:47] <keffo> lingering data trying to reconnect for one.
[Tuesday 07 September 2010] [07:33:20] <keffo> oh and more liberal means of implementing loadbalancing, but I’ve made that point already :)
[Tuesday 07 September 2010] [07:34:02] <pieterh> well, load balancing using XREP routing is pretty clear, and will be nicely explained in Ch3 of the Guide
[Tuesday 07 September 2010] [07:34:19] <keffo> will be? =)
[Tuesday 07 September 2010] [07:34:29] <pieterh> is in progress if I was not chatting here :-)
[Tuesday 07 September 2010] [07:34:39] <pieterh> anything to do with maintaining overall state is a different kettle of chicken, though
[Tuesday 07 September 2010] [07:34:45] <keffo> But I think I know what that will say by now :)
[Tuesday 07 September 2010] [07:35:05] <pieterh> hopefully, yeah
[Tuesday 07 September 2010] [07:35:41] <keffo> what I’m doing is something I’ve thought about for years though, so it should work :)
[Tuesday 07 September 2010] [07:35:55] <keffo> zmq solved a big gaping questionmark though :)
[Tuesday 07 September 2010] [07:36:43] <keffo> I might be getting a job soon though, so dev on this wll sadly be sidetracked to weekends and evenings only though :/
[Tuesday 07 September 2010] [07:37:32] <pieterh> is it open source?
[Tuesday 07 September 2010] [07:38:22] <keffo> it might be eventually!
[Tuesday 07 September 2010] [07:38:53] <keffo> would benefit the lua community I guess
[Tuesday 07 September 2010] [07:38:58] <pieterh> well... i’ve learned two relevant things here having done software for way too long
[Tuesday 07 September 2010] [07:39:04] <pieterh> a. if it’s not open source it will die
[Tuesday 07 September 2010] [07:39:17] <pieterh> b. if you don’t start as open source you can’t make it work afterwards
[Tuesday 07 September 2010] [07:40:02] <keffo> I dont agree, and I’ve never done anything other than software :)
[Tuesday 07 September 2010] [07:40:10] <pieterh> cause it’s not about building code but about building community...
[Tuesday 07 September 2010] [07:40:43] <pieterh> good luck, anyhow
[Tuesday 07 September 2010] [07:41:19] <keffo> I wasnt thinking of opensource as in leveraging resources, but simply aiding someone else, when I’m done with it :)
[Tuesday 07 September 2010] [07:41:49] <pieterh> nah, without people who helped make the code, it dies as soon as your evenings and weekends aren’t available any more
[Tuesday 07 September 2010] [07:42:34] <pieterh> it’s not about leveraging resources but about software that lives past the “free time” of its creator
[Tuesday 07 September 2010] [07:43:00] <pieterh> imho
[Tuesday 07 September 2010] [07:43:02] <keffo> Oh, it’s not free time at all, but I need to work for a bit to not starve :)
[Tuesday 07 September 2010] [07:43:33] <pieterh> starving is not pleasant, no
[Tuesday 07 September 2010] [07:44:20] <keffo> Not so much starving, but keeping girlfriend happier :)
[Tuesday 07 September 2010] [07:45:03] <keffo> when this is done, it will take me & the other dude about 2-3 weeks to produce the actual product we’ll eventually sell.. -That- part is already planned and so forth..
[Tuesday 07 September 2010] [07:45:17] <keffo> And ones that happens, there is no benefit to keep the code not opensource
[Tuesday 07 September 2010] [07:46:12] <keffo> err, once..
[Tuesday 07 September 2010] [07:46:53] <pieterh> :-) good spellng is hrad somtimes
[Tuesday 07 September 2010] [07:55:59] <keffo> um, this is odd
[Tuesday 07 September 2010] [07:56:10] <keffo> I wonder if lua might freak a little at some weird binary
[Tuesday 07 September 2010] [08:13:08] Nick
DasIch_ is now known as DasIch.
[Tuesday 07 September 2010] [08:50:26] Quit jsimmons has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [09:21:23] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Tuesday 07 September 2010] [09:51:25] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6d4ffd9 10/ (src/fq.cpp src/lb.cpp): Bug in fq_t and lb_t (when used via ZMQ_EVENTS option) fixed -
http://bit.ly/cvOPzL[Tuesday 07 September 2010] [10:12:19] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [10:43:46] Whois keffo is a user on channels: #zeromq
[Tuesday 07 September 2010] [10:43:46] Whois keffo is online via lindbohm.freenode.net (Stockholm, Sweden).
[Tuesday 07 September 2010] [10:43:46] Whois keffo has been idle for 2 hours, 47 minutes, and 36 seconds.
[Tuesday 07 September 2010] [10:43:46] Whois keffo has been online since 2010-09-07 02:18:32.
[Tuesday 07 September 2010] [10:43:46] Whois keffo is logged in as Keffo.
[Tuesday 07 September 2010] [10:43:46] Whois End of WHOIS list.
[Tuesday 07 September 2010] [11:14:02] <CIA-20> zeromq2: 03Martin Sustrik 07master * rf374431 10/ src/pipe.hpp : get rid of ‘has virtual functions but non-virtual destructor’ warnings in pipe.hpp -
http://bit.ly/9Relxm[Tuesday 07 September 2010] [11:21:12] <Tasser> cremes, it’s more about the ruby part you wrote
[Tuesday 07 September 2010] [11:21:36] <cremes> Tasser: i’m around if you have questions
[Tuesday 07 September 2010] [11:21:55] <Tasser> cremes, oh, just asking for the big picture
[Tuesday 07 September 2010] [11:22:03] <Tasser> aka what is where, how to stuff flows
[Tuesday 07 September 2010] [11:22:17] <cremes> sure...
[Tuesday 07 September 2010] [11:22:28] <Tasser> and probably write it down into your git
[Tuesday 07 September 2010] [11:22:35] <Tasser> HACKING or something like that :-)
[Tuesday 07 September 2010] [11:22:46] <cremes> whatever i write here, i’ll clean up and add to the README
[Tuesday 07 September 2010] [11:23:20] <cremes> ZM::Reactor is a thread that contains a single ZMQ context
[Tuesday 07 September 2010] [11:23:45] <cremes> from this context, you can create any kind of socket
[Tuesday 07 September 2010] [11:23:59] <cremes> (stop me if i’m not answering your question)
[Tuesday 07 September 2010] [11:24:21] <cremes> during socket creation, you pass a ruby object that will act as that socket’s handler
[Tuesday 07 September 2010] [11:24:32] <Tasser> callback?
[Tuesday 07 September 2010] [11:24:41] <cremes> the handler should provide on_attach, on_writable and on_readable methods
[Tuesday 07 September 2010] [11:25:32] <cremes> the on_attach method is called right away and lets you set things up (kind of like a constructor)
[Tuesday 07 September 2010] [11:25:47] <Tasser> so why not #new ?
[Tuesday 07 September 2010] [11:25:52] <cremes> the on_readable and on_writable methods are called when the socket is polled for those events and finds them to be true
[Tuesday 07 September 2010] [11:26:09] <cremes> explain what you mean by “not #new”?
[Tuesday 07 September 2010] [11:27:58] <Tasser> create a new instance per socket, so call #new and on that instance #on_writable, #on_readable
[Tuesday 07 September 2010] [11:29:26] <cremes> the handler instance is just a regular ruby class that implements the 3 methods i mentioned
[Tuesday 07 September 2010] [11:29:56] <cremes> it has a constructor (def initialize(
*args) nil; end) just like any other class
[Tuesday 07 September 2010] [11:30:16] <Tasser> less abstraction than EM
[Tuesday 07 September 2010] [11:30:21] <cremes> you could use one instance of a class to manage multiple sockets; look at the one-handed-ping-pong example
[Tuesday 07 September 2010] [11:30:33] <Tasser> yeah, having that one here atm
[Tuesday 07 September 2010] [11:30:36] <cremes> yeah, EM is kind of confusing with the EM::Connection stuff
[Tuesday 07 September 2010] [11:31:18] <Tasser> meh, gotta go :-(
[Tuesday 07 September 2010] [11:31:25] <cremes> sure
[Tuesday 07 September 2010] [11:31:50] <cremes> i’m usually on irc from 8am to 5pm central standard time (gmt -6, i think)
[Tuesday 07 September 2010] [11:32:08] <cremes> ping me if you have more questions or send them to the 0mq ml
[Tuesday 07 September 2010] [11:33:20] <bbigras> Is there a way to build zeromq with mingw?
[Tuesday 07 September 2010] [11:33:47] <cremes> bbigras: luislavena (rubyinstaller.org guy) has been playing with that
[Tuesday 07 September 2010] [11:34:14] <cremes> he opened an issue on github to fix a problem he encountered
[Tuesday 07 September 2010] [11:34:24] <cremes> so as far as i know he succeeded
[Tuesday 07 September 2010] [11:34:32] <bbigras> cremes: nice, thanks!
[Tuesday 07 September 2010] [11:35:16] <bbigras> cremes: Do you know if anyone had success using zeromq with Qt without having to disable Qt’s signal/slot macros?
[Tuesday 07 September 2010] [11:36:11] <cremes> i haven’t heard anything about that, so no
[Tuesday 07 September 2010] [11:36:23] <cremes> you might try asking on the 0mq ML
[Tuesday 07 September 2010] [11:38:07] <bbigras> cremes: thanks
[Tuesday 07 September 2010] [12:08:37] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.8/20100722155716]).
[Tuesday 07 September 2010] [12:11:23] Quit plq has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [12:20:59] Quit xla has left this server (Quit: leaving).
[Tuesday 07 September 2010] [12:36:25] Quit mato has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [12:43:48] Quit mato has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [12:54:09] Quit gebi has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [13:08:12] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 07 September 2010] [13:47:19] Quit keffo has left this server (Ping timeout: 252 seconds).
[Tuesday 07 September 2010] [13:50:51] Quit psino has left this server (Quit: psino).
[Tuesday 07 September 2010] [14:12:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [14:17:07] <ModusPwnens> hi
[Tuesday 07 September 2010] [14:17:50] <ModusPwnens> Is there anyone in here that has used google protobufs with zeromq? I’m wondering what kind of throughput is normal when using google protobufs
[Tuesday 07 September 2010] [14:19:03] Quit psino has left this server (Changing host).
[Tuesday 07 September 2010] [14:20:19] <cremes> ModusPwnens: i recommend you write a small benchmark that serializes/deserializes your data
[Tuesday 07 September 2010] [14:20:31] <cremes> and see what the upper limit is on your message rate
[Tuesday 07 September 2010] [14:20:48] <cremes> then 0mq will have that as its upper limit for throughput
[Tuesday 07 September 2010] [14:28:16] Quit dos000 has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [14:41:24] <ModusPwnens> yeah i did this cremes
[Tuesday 07 September 2010] [14:41:40] <ModusPwnens> I am just wondering whether or not my results are expected
[Tuesday 07 September 2010] [14:42:11] <ModusPwnens> Hmm, well actually, I benchmarked it with zeromq too
[Tuesday 07 September 2010] [14:42:30] <ModusPwnens> so it’s timing how long it takes to send messages as well
[Tuesday 07 September 2010] [14:44:04] <cremes> ModusPwnens: yeah, take 0mq out of the equation to get an upper bound
[Tuesday 07 September 2010] [14:44:17] <cremes> then you can test with 0mq to see what kind of overhead it is introducing
[Tuesday 07 September 2010] [14:57:44] <ModusPwnens> Hmm, another thing, I am having somewhat surprising results with the remote/local throughput tests
[Tuesday 07 September 2010] [14:58:27] <ModusPwnens> I am just using localhost, but I only get around 200 Mb/s throughput, which seems low to me.
[Tuesday 07 September 2010] [14:59:26] <cremes> ModusPwnens: try increasing the message body size from 50 bytes (from the example you posted last week) to something larger
[Tuesday 07 September 2010] [14:59:44] <cremes> also, note that the remote/local tests are doing a ping pong with REQ/REP sockets
[Tuesday 07 September 2010] [15:00:06] <cremes> you could see higher throughput on a PUB socket
[Tuesday 07 September 2010] [15:01:07] <ModusPwnens> Yeah, I have tried with larger message sizes. 5000 byte and 2500 count messages
[Tuesday 07 September 2010] [15:01:38] <ModusPwnens> is 200mb/s normal on localhost? I would have thought it would be much much faster
[Tuesday 07 September 2010] [15:01:43] <cremes> and what did you see?
[Tuesday 07 September 2010] [15:01:50] <cremes> with the varying message sizes...?
[Tuesday 07 September 2010] [15:02:20] <ModusPwnens> I see about 200mb/s or 5000 messages/s
[Tuesday 07 September 2010] [15:02:35] <cremes> so you see the same throughput regardless of message size?
[Tuesday 07 September 2010] [15:03:24] <ModusPwnens> not really, when the parameters are smaller I see different values
[Tuesday 07 September 2010] [15:03:41] <ModusPwnens> when i said 200 earlier i was using these paremeters
[Tuesday 07 September 2010] [15:03:56] <cremes> so what does 200 MB/s represent? the best that you see, the average or the worst?
[Tuesday 07 September 2010] [15:04:04] <ModusPwnens> about the average
[Tuesday 07 September 2010] [15:04:10] <ModusPwnens> i see 180 sometimes, sometimes 220
[Tuesday 07 September 2010] [15:04:33] <cremes> this is on windows, right?
[Tuesday 07 September 2010] [15:04:45] <ModusPwnens> Yeah. I was wondering if it would be faster on linux
[Tuesday 07 September 2010] [15:05:10] <cremes> sometimes; it’s hard to draw conclusions because this stuff is so dependent on OS and the hardware
[Tuesday 07 September 2010] [15:05:27] <ModusPwnens> Yeah. I notice that the official results on the zeromq website are ridiculously high
[Tuesday 07 September 2010] [15:05:38] <ModusPwnens> but I’m thinking that’s because they have a godly computer with 12 gigs of ram and 4 cores
[Tuesday 07 September 2010] [15:05:59] <ModusPwnens> i only have 1 core on this computer
[Tuesday 07 September 2010] [15:06:13] <ModusPwnens> as well as only 3 gigs of ram
[Tuesday 07 September 2010] [15:06:25] <icy> size of ram does not matter, speed does
[Tuesday 07 September 2010] [15:06:33] <ModusPwnens> I know that has an effect, but I’m not sure how large an effect that would be.
[Tuesday 07 September 2010] [15:06:36] <cremes> that computer is ancient if it has only 1 core; i don’t think intel has shipped a 1-core desktop cpu since around 2006
[Tuesday 07 September 2010] [15:07:09] <ModusPwnens> it’s actually relatively new
[Tuesday 07 September 2010] [15:07:13] <ModusPwnens> amd sempron m100
[Tuesday 07 September 2010] [15:07:22] <ModusPwnens> which i think has only 1 core
[Tuesday 07 September 2010] [15:08:47] <cremes> amd is behind the curve; sorry :)
[Tuesday 07 September 2010] [15:09:11] <ModusPwnens> Heh, apparently.
[Tuesday 07 September 2010] [15:09:18] <ModusPwnens> Anyways, is that sort of throughput expected?
[Tuesday 07 September 2010] [15:09:36] <cremes> again, it’s dependent upon OS and hardware
[Tuesday 07 September 2010] [15:09:49] <ModusPwnens> Hmm. So it’s at least not abnormal?
[Tuesday 07 September 2010] [15:10:10] <cremes> if you have nothing to compare it to...?
[Tuesday 07 September 2010] [15:10:43] <ModusPwnens> Ah, well there you go i guess.
[Tuesday 07 September 2010] [15:11:23] <icy> well, what cpu usage do you get while benchmarking?
[Tuesday 07 September 2010] [15:11:51] <icy> maybe it’s just really slow ram :)
[Tuesday 07 September 2010] [15:12:35] <ModusPwnens> i get 100% CPU usage
[Tuesday 07 September 2010] [15:13:24] <ModusPwnens> Does that mean the RAM is just slow?
[Tuesday 07 September 2010] [15:13:41] <Steve-o> what parameters are you using, I can compare with a single core Xeon right now
[Tuesday 07 September 2010] [15:13:46] <ModusPwnens> ok
[Tuesday 07 September 2010] [15:13:50] <ModusPwnens> for the cpu usage test I just ran
[Tuesday 07 September 2010] [15:13:57] <ModusPwnens> I used
[Tuesday 07 September 2010] [15:14:01] <ModusPwnens> 5000 byte messages
[Tuesday 07 September 2010] [15:14:08] <ModusPwnens> and 250,000 message count
[Tuesday 07 September 2010] [15:14:17] <ModusPwnens> all on localhost
[Tuesday 07 September 2010] [15:15:07] <Steve-o> I get 39,031 msgs/s and 1561 Mb/s
[Tuesday 07 September 2010] [15:15:10] <guido_g> message size: 1024 [B]
[Tuesday 07 September 2010] [15:15:10] <guido_g> message count: 100000
[Tuesday 07 September 2010] [15:15:10] <guido_g> mean throughput: 381327 [msg/s]
[Tuesday 07 September 2010] [15:15:10] <guido_g> mean throughput: 3123.831 [Mb/s]
[Tuesday 07 September 2010] [15:15:22] <guido_g> also a notebook
[Tuesday 07 September 2010] [15:16:25] <ModusPwnens> Hmm, that is definitely much higher than what I am getting.
[Tuesday 07 September 2010] [15:16:38] <ModusPwnens> how much ram do you have and what kind of processor?
[Tuesday 07 September 2010] [15:16:50] <ModusPwnens> guido that is
[Tuesday 07 September 2010] [15:16:53] <guido_g> Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz
[Tuesday 07 September 2010] [15:17:12] <guido_g> ram usage is not a problem at all
[Tuesday 07 September 2010] [15:17:35] <guido_g> it’s more a matter of cache and latency
[Tuesday 07 September 2010] [15:17:56] <ModusPwnens> Latency shouldn’t really be a problem on localhost though...right?
[Tuesday 07 September 2010] [15:18:16] <guido_g> ram latency
[Tuesday 07 September 2010] [15:18:20] <guido_g> sigh
[Tuesday 07 September 2010] [15:19:06] <icy> and bandwidth
[Tuesday 07 September 2010] [15:19:19] <ModusPwnens> Oh..sorry about that, i misunderstood..
[Tuesday 07 September 2010] [15:19:58] <guido_g> message size: 1024 [B]
[Tuesday 07 September 2010] [15:19:58] <guido_g> message count: 10000
[Tuesday 07 September 2010] [15:19:58] <guido_g> mean throughput: 98001 [msg/s]
[Tuesday 07 September 2010] [15:19:58] <guido_g> mean throughput: 802.824 [Mb/s]
[Tuesday 07 September 2010] [15:20:01] <guido_g> via lan
[Tuesday 07 September 2010] [15:20:21] <guido_g> so obviously you’re on a dog slow machine
[Tuesday 07 September 2010] [15:20:47] <icy> or windows :P
[Tuesday 07 September 2010] [15:21:15] <guido_g> hrhrhr
[Tuesday 07 September 2010] [15:22:47] <icy> nice webserver choice for zeromq.org
[Tuesday 07 September 2010] [15:23:15] <ModusPwnens> Hmm thanks. I will try to procure another computer to test this on.
[Tuesday 07 September 2010] [15:23:49] <guido_g> icy: you mean wikidot?
[Tuesday 07 September 2010] [15:24:13] <icy> I mean the lighttpd part :)
[Tuesday 07 September 2010] [15:31:34] <icy> hm local_thr does not seem to do anything on my osx box
[Tuesday 07 September 2010] [15:47:59] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [15:49:05] <cremes> icy: you need to run local_thr and remote_thr as a pair; one is the client and the other is the server
[Tuesday 07 September 2010] [15:49:16] <icy> ah right, thx
[Tuesday 07 September 2010] [15:53:30] <icy> ouf, this thing just sent me 1gb into swap
[Tuesday 07 September 2010] [15:54:15] <icy> sending 1000000 1kb messages
[Tuesday 07 September 2010] [15:54:35] <icy> I guess they get buffered in ram
[Tuesday 07 September 2010] [15:57:29] <cremes> icy: the receiver must not have pulled them off the queue fast enough
[Tuesday 07 September 2010] [15:57:43] <cremes> when i run those tests on my system, memory usage is constant (no queueing)
[Tuesday 07 September 2010] [15:58:08] <cremes> why don’t you pastie the arguments you passed to both programs so we can comment
[Tuesday 07 September 2010] [15:59:54] <icy> maybe I should start local_thr before remote_thr :)
[Tuesday 07 September 2010] [16:00:02] <cremes> try lowering the 1 million to 10 thousand and monitor the memory size of the programs
[Tuesday 07 September 2010] [16:00:22] <cremes> yeah, start order is important...
[Tuesday 07 September 2010] [16:01:08] <icy> doing that I get it to work even though it hits into swap briefly (understandable, the sender will always be faster)
[Tuesday 07 September 2010] [16:02:21] <cremes> icy: not true; this test is using REQ/REP sockets so it should only have 1 message in flight at any given time
[Tuesday 07 September 2010] [16:02:33] <cremes> one sender should not be able to get ahead of the other
[Tuesday 07 September 2010] [16:03:21] <cremes> (i was wrong in my statement from 2:57; no queueing should occur)
[Tuesday 07 September 2010] [16:03:22] <icy> I get 40mb ram usage with 100k messages
[Tuesday 07 September 2010] [16:03:45] <cremes> are you running the C programs or using the samples from another language binding?
[Tuesday 07 September 2010] [16:04:38] <icy> perf/ <- the ones in there which are C I think
[Tuesday 07 September 2010] [16:05:03] <cremes> did you modify the code at all?
[Tuesday 07 September 2010] [16:05:37] <icy> no, downloaded tarball, ./configure, make and ran the apps
[Tuesday 07 September 2010] [16:05:49] <cremes> huh... what OS?
[Tuesday 07 September 2010] [16:05:53] <icy> osx
[Tuesday 07 September 2010] [16:06:03] <cremes> 2.0.9?
[Tuesday 07 September 2010] [16:06:14] <cremes> 0mq, that is
[Tuesday 07 September 2010] [16:06:19] <icy> yea
[Tuesday 07 September 2010] [16:06:27] <Samy> I see a lot of emphasis on lock-free algorithms on the ZeroMQ website.
[Tuesday 07 September 2010] [16:06:28] <cremes> weird
[Tuesday 07 September 2010] [16:06:36] <cremes> it should not have unbounded memory growth
[Tuesday 07 September 2010] [16:06:42] <cremes> you should file a bug
[Tuesday 07 September 2010] [16:06:42] <Samy> What lock-free objects does ZeroMQ use? Where would the source-code be for them?
[Tuesday 07 September 2010] [16:07:02] <Samy> The atomics interface seemed too simple to support more complex data structures, though I wasn’t looking at the right thing.
[Tuesday 07 September 2010] [16:07:21] <cremes> Samy: check out the y_pipe stuff... i believe that is where the lock-free algorithms are used though sustrik would know better (he wrote it)
[Tuesday 07 September 2010] [16:07:43] <Samy> cremes, cool. Does sustrik IRC?
[Tuesday 07 September 2010] [16:08:05] <cremes> Samy: yes... he’s usually in channel but he isn’t here right now
[Tuesday 07 September 2010] [16:08:21] <Samy> Ok, thank you.
[Tuesday 07 September 2010] [16:08:34] <icy> the real mem does not show all the memory it uses as I’m already several hundred mb into swap at the time the screenshot was made
[Tuesday 07 September 2010] [16:08:55] <cremes> icy: that shouldn’t be; i would open a bug and describe the problem
[Tuesday 07 September 2010] [16:09:09] <cremes> make sure to include 0mq release, OS, OS release, etc
[Tuesday 07 September 2010] [16:09:19] <icy> k
[Tuesday 07 September 2010] [16:10:47] <cremes> icy: hold on a sec...
[Tuesday 07 September 2010] [16:11:07] <cremes> were you doing local_thr or local_lat as your perf test?
[Tuesday 07 September 2010] [16:14:00] <icy> thr
[Tuesday 07 September 2010] [16:16:59] <cremes> the local_thr/remote_thr examples don’t make any sense
[Tuesday 07 September 2010] [16:17:19] <cremes> the remote_thr program is using a REQ socket while the local_thr is using a SUB socket
[Tuesday 07 September 2010] [16:17:23] <cremes> the two are not compatible
[Tuesday 07 September 2010] [16:18:52] <icy> I’m totally knew to zeromq, just saw this benchmark app in the src dir and thought I’d give it a go :)
[Tuesday 07 September 2010] [16:18:55] <cremes> nm... i’m looking at the wrong stuff
[Tuesday 07 September 2010] [16:18:57] <cremes> argh...
[Tuesday 07 September 2010] [16:19:22] <cremes> okay, so remote_thr uses a PUB socket while local_thr uses a SUB
[Tuesday 07 September 2010] [16:19:24] <cremes> that is correct
[Tuesday 07 September 2010] [16:19:29] <cremes> you need to start local_thr first
[Tuesday 07 September 2010] [16:19:50] <cremes> remote_thr will slam your system by publishing as fast as possible, so there will be queueing
[Tuesday 07 September 2010] [16:20:05] <cremes> (i was thinking of the local_lat/remote_lat examples which uses different socket types)
[Tuesday 07 September 2010] [16:20:48] <icy> I did start local_thr first and even after that execited, remote_thr was allocating ram
[Tuesday 07 September 2010] [16:21:09] <icy> s/execited/exited/
[Tuesday 07 September 2010] [16:21:20] <cremes> yeah, i’m looking at it now...
[Tuesday 07 September 2010] [16:23:20] <cremes> if everything is working correctly, remote_thr should exit first
[Tuesday 07 September 2010] [16:37:31] Quit cremes has left this server (Quit: cremes).
[Tuesday 07 September 2010] [16:44:32] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Tuesday 07 September 2010] [16:45:13] Quit cremes has left this server (Client Quit).
[Tuesday 07 September 2010] [16:45:21] Part andrewvc has left this channel.
[Tuesday 07 September 2010] [16:46:33] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Tuesday 07 September 2010] [16:55:32] <dermoth> I’m wondering if there’s an easy way to monitor a queue size on a zeromq worker... It doesn’t seems like there is anything in the API
[Tuesday 07 September 2010] [16:55:42] <dermoth> on a zeromq broker I mean
[Tuesday 07 September 2010] [16:55:46] <dermoth> i.e. a device
[Tuesday 07 September 2010] [17:02:24] <ModusPwnens> whoa
[Tuesday 07 September 2010] [17:02:31] <ModusPwnens> im getting a strange error with the benchmarking tests now
[Tuesday 07 September 2010] [17:04:44] <dermoth> my concern is that the PUSH workers may send more messages than can be processed by the PULL workers, eventually filling up the queue. This would be possible if the PULL workers sync their state to disk to avoid data loss...
[Tuesday 07 September 2010] [17:12:07] <ModusPwnens> hey does zeroMQ allocate memory for messages all at once?
[Tuesday 07 September 2010] [17:12:25] Quit plq has left this server (Quit: Leaving).
[Tuesday 07 September 2010] [17:13:35] <ModusPwnens> Or maybe it is because I am using the publish subscribe topology so it just continuously creates messages and the receiving end is not fast enough..
[Tuesday 07 September 2010] [17:33:14] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 07 September 2010] [18:23:07] <cremes> ModusPwnens: yes to the second thing you said; the publisher outpaces the subscriber
[Tuesday 07 September 2010] [18:23:34] <cremes> dermoth: no, there is no way to fetch the queue size; check the mailing list for the reasons why
[Tuesday 07 September 2010] [18:23:48] <cremes> that topic has been raised and answered a bunch of times (someone should add it to the FAQ)
[Tuesday 07 September 2010] [18:24:15] <cremes> dermoth: also, check out HWM (high water mark) settings for the PUSH sockets
[Tuesday 07 September 2010] [18:24:53] <cremes> by setting HWM, it will block when the queue hits that message level (or return EAGAIN if you try sending with ZMQ_NOBLOCK)
[Tuesday 07 September 2010] [18:26:05] <dermoth> yes, but I need to know berore i’ll ve blocking... I guess I could test the latency though
[Tuesday 07 September 2010] [18:26:34] <cremes> dermoth: what do you mean that you need to know before?
[Tuesday 07 September 2010] [18:26:44] <cremes> and what does latency have to do with it?
[Tuesday 07 September 2010] [18:27:21] <dermoth> well, if my queues start filling up at peak times I want to react before all pushers block... my primary goal in using zmq is to avoid blocking
[Tuesday 07 September 2010] [18:28:04] <cremes> ok, then use send with ZMQ_NOBLOCK and test for EAGAIN; when you get it then you know you have hit your high water mark
[Tuesday 07 September 2010] [18:28:14] <cremes> and you can take whatever action is necessary
[Tuesday 07 September 2010] [18:29:03] <dermoth> cremes, if the queue fill up on the device, then there will be some latency between the time I push to the queue and the time my worked gets the event. I can push a special message and have whichever worker gets it respond to me, then I know the latency. it it rises thern I need more workers downstream
[Tuesday 07 September 2010] [18:29:53] <cremes> ok
[Tuesday 07 September 2010] [18:30:25] <cremes> you could also have another pair of sockets where each worker tells the server/pusher that it has received a message
[Tuesday 07 September 2010] [18:30:54] <cremes> using this “out of band” communication, you could publish only those messages that can be immediately handled by a worker
[Tuesday 07 September 2010] [18:31:25] <cremes> you would have at most 1 message in someone’s queue because you would not push another one until each one had been acknowledged
[Tuesday 07 September 2010] [18:31:48] <cremes> that seems better than trying to rely on some weird latency calculation that might not be trustworthy
[Tuesday 07 September 2010] [18:31:51] <dermoth> i’ll implement XREQ/XREP for thing I need to make sure a worker is getting it... the rest is high-throghtput stuff that can suffer a small percentage loss...
[Tuesday 07 September 2010] [18:32:28] <cremes> definitely take a look at HWM
[Tuesday 07 September 2010] [18:32:36] <cremes> i think it does what you need
[Tuesday 07 September 2010] [18:33:27] <dermoth> the point is that I don’t want to block on the sending side... HWM will be useful in logging error conditions, but I should never end up hitting this limit...
[Tuesday 07 September 2010] [18:34:21] <cremes> dermoth: sorry, but your requirements don’t make sense to me
[Tuesday 07 September 2010] [18:34:59] <cremes> if you could get the queue length, you would probably prevent your pusher from sending more messages if it hit some threshold, right?
[Tuesday 07 September 2010] [18:35:08] <cremes> if so, then this is exactly what you can do with HWM
[Tuesday 07 September 2010] [18:35:27] <cremes> couple HWM with NO_BLOCK and you’ll get your “signal” that there aren’t enough workers
[Tuesday 07 September 2010] [18:35:38] <dermoth> well I don’t really control the pusher... it logs as data comes it. I need to have enough pullers to absorb the data as it comes in
[Tuesday 07 September 2010] [18:35:38] <cremes> assuming you ever hit the HWM
[Tuesday 07 September 2010] [18:36:28] <cremes> so you don’t have any control over throttling the pusher?
[Tuesday 07 September 2010] [18:37:24] <dermoth> because if the pushers block, it’s not goung to work anymore and I will loose data - the point is that the pushhers have to be non-blocking. But i’ll figure out something... maybe even measuring the rss size of the process might work.
[Tuesday 07 September 2010] [18:37:54] <cremes> dermoth: let me say it again then.... use send with NO_BLOCK and test for EAGAIN
[Tuesday 07 September 2010] [18:37:58] <cremes> this will NOT BLOCK
[Tuesday 07 September 2010] [18:38:05] <dermoth> yes I get that ;)
[Tuesday 07 September 2010] [18:38:20] <cremes> so what’s the problem then? ;)
[Tuesday 07 September 2010] [18:39:53] <dermoth> what I mean, EAGIN is bad too - I will log it & alert on it, but I want to know before that happens... If I start getting latency cpikes when I turn worn downstream workers, or during pikes, even before filling up the queues all the way up to the pushers, I want to know it. But that’s fine, I’ll work with what I have ;)
[Tuesday 07 September 2010] [18:40:16] <cremes> ok, i see
[Tuesday 07 September 2010] [18:40:49] <cremes> let us know what you come up with or add your solution to the wiki
[Tuesday 07 September 2010] [18:41:13] <dermoth> Sure. btw, it having multiple parallel devices wiorking well? i.e. if one crash will it nicely fall back to the other one? Will my XREQ/XREP suffer any latency? More that 2-3 seconds avg per requests may become problematic
[Tuesday 07 September 2010] [18:42:04] <dermoth> here actually on the ends it more like a REQ/REP socket, byt the device will be XREQ/XREP to pass & load-balance the messages
[Tuesday 07 September 2010] [18:43:32] <cremes> a device is just a fancy packaging for 2 sockets to aid with load balancing
[Tuesday 07 September 2010] [18:43:46] <cremes> if you are worried about a device crashing, then messages in flight could be lost
[Tuesday 07 September 2010] [18:43:49] <dermoth> i’ll probably be running some tests anyway... Thanks
[Tuesday 07 September 2010] [18:44:23] <cremes> if you can’t handle data loss, then you need to add some code around all of this to ack/nak messages and retry when things timeout or disappear
[Tuesday 07 September 2010] [18:44:34] <cremes> none of that is built in to 0mq; you have to build it on top
[Tuesday 07 September 2010] [18:44:57] <dermoth> yes... a srever crashing is pretty rare, what i’m concerned about it if the system will keep on working. the REQ/REP are for where I can’t handle loss and I will loose the request if the device crash with my message
[Tuesday 07 September 2010] [18:45:07] <cremes> and unless you have a really slow link or are sending very large messages, i can’t imagine it would ever take 2-3 seconds to send a message
[Tuesday 07 September 2010] [18:45:38] <cremes> if you use XREQ/XREP and do explicit acks, then you could have things continue to work
[Tuesday 07 September 2010] [18:46:12] <cremes> if you use REQ/REP sockets, they enforce a very strict send/recv/send/recv pattern, so a crashed peer would not be recoverable
[Tuesday 07 September 2010] [18:46:21] <dermoth> well, i’m thinking possible tcp timeouts, etc. I would assube ZeroMQ is able to handle well peers, but I will test it anyway. Thanks!
[Tuesday 07 September 2010] [18:46:31] <cremes> ok, good luck
[Tuesday 07 September 2010] [18:46:50] <dermoth> to handle well down peers
[Tuesday 07 September 2010] [18:47:44] <cremes> dermoth: TCP timeouts are not exposed to you via the 0mq api
[Tuesday 07 September 2010] [18:48:15] <cremes> 0mq may see a connection disappear at the tcp level, but that doesn’t mean that the socket is no good
[Tuesday 07 September 2010] [18:48:31] <cremes> because a 0mq socket can be bound or connected to multiple endpoints
[Tuesday 07 September 2010] [18:48:50] <cremes> one endpoint failure does not cause the whole 0mq socket to fail
[Tuesday 07 September 2010] [18:49:28] <cremes> the 0mq socket will continue to load balance messages to any “surviving” endpoints
[Tuesday 07 September 2010] [18:51:34] <dermoth> ok, sounds good. FWIW, what I have in mind to mointor PUSH/PULL latency is to send a special message with an IP/port to reply withm and have the workers respond with an udp packet. I can release the check as a Nagios plugin like I usually do for my monitoring scripts, but obviously the other end is up to the developer to get right
[Tuesday 07 September 2010] [18:53:32] <cremes> sounds neat
[Tuesday 07 September 2010] [19:06:05] Quit Steve-o has left this server (Ping timeout: 252 seconds).
[Tuesday 07 September 2010] [19:07:38] Quit keffo has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [19:09:50] Quit GeekGod has left this server (Quit: GeekGod).
[Tuesday 07 September 2010] [19:22:50] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 07 September 2010] [19:31:47] Quit psino has left this server (Quit: psino).
[Tuesday 07 September 2010] [20:37:05] Quit DasIch has left this server (Ping timeout: 260 seconds).
[Tuesday 07 September 2010] [21:11:23] Quit ModusPwnens has left this server (Ping timeout: 245 seconds).
[Tuesday 07 September 2010] [21:46:30] Quit gebi has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [22:25:30] Quit
halfie_ has left this server (Ping timeout: 240 seconds).
[Tuesday 07 September 2010] [23:47:58] Quit miguelito has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [02:41:08] Quit ivan has left this server (Ping timeout: 258 seconds).
[Wednesday 08 September 2010] [03:07:00] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [03:07:12] Quit
halfie_ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [04:12:25] <CIA-20> zeromq2: 03Martin Sustrik 07master * r91ea204 10/ (13 files in 2 dirs): EINTR returned from the blocking functions -
http://bit.ly/aSdLhb[Wednesday 08 September 2010] [04:29:31] <lestrrat> sustrik: almost.... s/Daisuke Make/Daisuke Maki/ ;) But yes, thanks for the EINTR fix!
[Wednesday 08 September 2010] [04:39:46] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 08 September 2010] [04:47:44] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [04:53:08] Quit jsimmons has left this server (Quit: Leaving).
[Wednesday 08 September 2010] [05:00:48] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 08 September 2010] [05:25:30] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 08 September 2010] [06:33:47] Quit gebi has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [07:15:44] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 08 September 2010] [07:44:46] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [07:58:48] <keffo> :)
[Wednesday 08 September 2010] [08:05:09] Quit __moore__ has left this server (Read error: Operation timed out).
[Wednesday 08 September 2010] [08:07:28] <pieterh> keffo: if I can figure out wtf I can do with an XREP talking to an XREP, I’ll finish this part of the Guide today
[Wednesday 08 September 2010] [08:07:44] <pieterh> turns out least-recently used is indecently easy to do
[Wednesday 08 September 2010] [08:11:00] <keffo> That would ameliorate at least part of the problem :)
[Wednesday 08 September 2010] [08:11:40] <keffo> Except not ‘successful send’ in my case, but when the answer is returned :)
[Wednesday 08 September 2010] [08:12:36] <keffo> It wouldn’t be very difficult for me to specify routing manually though, bypassing the zmq lb altogether
[Wednesday 08 September 2010] [08:12:46] <keffo> but I need to finish heartbeat system first
[Wednesday 08 September 2010] [08:13:18] <pieterh> well, here’s my take on custom LB...
[Wednesday 08 September 2010] [08:13:18] <keffo> I’m thinking pub -> sub->xreq back to source, and that would give me both route & availability?
[Wednesday 08 September 2010] [08:13:33] <pieterh> you want to use a XREP socket to do the routing
[Wednesday 08 September 2010] [08:13:41] <pieterh> you can use various algorithms to drive it
[Wednesday 08 September 2010] [08:13:52] <pieterh> depending on the socket type it talks to
[Wednesday 08 September 2010] [08:14:26] <keffo> I’m thinking a priorityqueue of all known/active nodes, then ‘nudging’ the weights based on speed/performance
[Wednesday 08 September 2010] [08:14:54] <pieterh> e.g. if your workers use req sockets, you get a nice LRU
[Wednesday 08 September 2010] [08:15:14] <pieterh> you don’t need a priority queue, just the message from the worker saying “ready”
[Wednesday 08 September 2010] [08:15:31] <keffo> hum, explain how req gives lru?
[Wednesday 08 September 2010] [08:15:43] <pieterh> well, think of HTTP long poll
[Wednesday 08 September 2010] [08:15:59] <pieterh> flow is worker says “ready”, router provides it workload
[Wednesday 08 September 2010] [08:16:09] <pieterh> workers say “ready” when they’re done...
[Wednesday 08 September 2010] [08:16:21] <pieterh> ready messages come into queue, router takes one off each time
[Wednesday 08 September 2010] [08:20:39] <keffo> uh, must draw on paper..
[Wednesday 08 September 2010] [08:21:03] <pieterh> hey, let me post this section of Ch 3, see if it makes any sense at all
[Wednesday 08 September 2010] [08:21:06] <pieterh> :-)
[Wednesday 08 September 2010] [08:21:19] <keffo> I think I need two prioqueues to be honest, one for picking the best of available set of nodes, and one for handing the most urgent tasks
[Wednesday 08 September 2010] [08:21:51] <keffo> let me know when you finish it!
[Wednesday 08 September 2010] [08:22:47] <guido_g> playing the “naming awareness” card: this isn’t REQuest/REPly anymore then, right?
[Wednesday 08 September 2010] [08:24:31] <pieterh> nope, this ain’t kansas anymore
[Wednesday 08 September 2010] [08:24:44] <pieterh> this is “off-road”...
[Wednesday 08 September 2010] [08:27:12] <pieterh> basically it’s about using identities and constructing envelopes so that XREP will route as we want to other nodes
[Wednesday 08 September 2010] [08:27:53] <pieterh> I just posted what I have...
[Wednesday 08 September 2010] [08:31:23] Quit rossij has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [08:33:13] <keffo> looks good
[Wednesday 08 September 2010] [08:35:28] <keffo> except the pub/sub ctrl+c sample code needs linebreaks
[Wednesday 08 September 2010] [08:37:57] <keffo> hum, it gets progressively difficult to read after that :)
[Wednesday 08 September 2010] [08:37:58] <pieterh> what sample is that?
[Wednesday 08 September 2010] [08:38:03] <pieterh> :-)
[Wednesday 08 September 2010] [08:38:35] <keffo> something got weird halfway down the article, it’s still embedded in the code preview
[Wednesday 08 September 2010] [08:38:45] <pieterh> hang on, refresh the page
[Wednesday 08 September 2010] [08:38:55] <pieterh> there was a fault in the text, i already fixed it
[Wednesday 08 September 2010] [08:39:16] <keffo> ignore, reloaded :)
[Wednesday 08 September 2010] [08:39:28] <keffo> lots of good food for thought there
[Wednesday 08 September 2010] [08:39:37] <guido_g> ack
[Wednesday 08 September 2010] [08:39:44] <guido_g> fig 22, the dashes are misplaced
[Wednesday 08 September 2010] [08:40:09] * keffo bbll
[Wednesday 08 September 2010] [08:40:10] <pieterh> guido_g: this is ditaa being too clever
[Wednesday 08 September 2010] [08:40:21] <pieterh> it thinks ‘-‘ in text is me trying to draw a line
[Wednesday 08 September 2010] [08:40:33] <guido_g> ouch
[Wednesday 08 September 2010] [08:40:49] <pieterh> ill try a unicode n-dash or something
[Wednesday 08 September 2010] [08:40:59] <guido_g> so no dash between single and hop and a slash for req/rep :)
[Wednesday 08 September 2010] [08:41:40] <pieterh> it’s always “request-reply”...
[Wednesday 08 September 2010] [08:42:09] <guido_g> ahh... then...
[Wednesday 08 September 2010] [08:42:40] <pieterh> np, easy to fix this in the parser
[Wednesday 08 September 2010] [08:45:52] <pieterh> bah, ditaa does not like Unicode at all, just dies
[Wednesday 08 September 2010] [08:45:53] <pieterh> well, too bad
[Wednesday 08 September 2010] [08:55:20] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [08:55:47] <guido_g> no escape mechanism?
[Wednesday 08 September 2010] [08:57:30] <pieterh> no mechanisms of any kind afaics
[Wednesday 08 September 2010] [08:57:45] <guido_g> oh
[Wednesday 08 September 2010] [08:57:50] <pieterh> which is ok, I’ll email the author
[Wednesday 08 September 2010] [09:16:50] Quit
keffo_ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [09:46:13] <CIA-20> zeromq2: 03Martin Sustrik 07master * r47e87b7 10/ include/zmq.h : EMTHREAD error code returned to zmq.h to ensure backward compatibility -
http://bit.ly/dtznl6[Wednesday 08 September 2010] [10:07:26] <sd88g93> greetings !
[Wednesday 08 September 2010] [10:08:44] <sd88g93> when are the zmq_msg_move() zmq_msg_copy() functions meant to be used ? i mean, what situations call for them ?
[Wednesday 08 September 2010] [10:14:57] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [10:27:50] <cremes> sd88g93: whenever you send data, the library takes ownership of your zmq_msg_t and calls close() on it
[Wednesday 08 September 2010] [10:27:59] <cremes> on the recv side, you are responsible for its lifecycle
[Wednesday 08 September 2010] [10:28:35] <cremes> so you might want to make a copy (which i think does reference counting internally) so that you could send it without it being deallocated
[Wednesday 08 September 2010] [10:28:47] <cremes> the copy is cheap (ref counting)
[Wednesday 08 September 2010] [10:28:56] <cremes> i don’t really know of a use case for move
[Wednesday 08 September 2010] [10:29:14] <sd88g93> cremes: what about VSM’s, where the message is embedded right in the msg struct ?
[Wednesday 08 September 2010] [10:29:23] <cremes> i don’t know
[Wednesday 08 September 2010] [10:29:32] <sd88g93> ohok
[Wednesday 08 September 2010] [10:29:35] <cremes> i’d have to read the code to figure out what happens
[Wednesday 08 September 2010] [10:30:06] <sd88g93> well, what about if you are recieving messages of a multi part message and relaying them through inproc to another thread ?
[Wednesday 08 September 2010] [10:30:19] <sd88g93> is it alright to just recieve and send the same msg ?
[Wednesday 08 September 2010] [10:30:30] <cremes> yes
[Wednesday 08 September 2010] [10:30:48] <cremes> recv does not automatically call close on the message, only sending does
[Wednesday 08 September 2010] [10:30:51] <sd88g93> i’m recieving a multipart message by way of a req/rep socket, and want to relay them to ap ublisher
[Wednesday 08 September 2010] [10:31:32] <sd88g93> but i’m relaying them first through inproc to the main thread, and then forwarding them to the one pub skt
[Wednesday 08 September 2010] [10:31:44] <cremes> oh, so two hops?
[Wednesday 08 September 2010] [10:31:45] <sd88g93> the inproc is failing, and just trying to figure out why
[Wednesday 08 September 2010] [10:31:58] <sd88g93> yeah, 2 hops
[Wednesday 08 September 2010] [10:32:28] <cremes> well, i’m not sure... i haven’t played much with inproc transports
[Wednesday 08 September 2010] [10:32:44] <cremes> as an optimization it might be doing ref counting on the message so that it doesn’t have to copy it for real
[Wednesday 08 September 2010] [10:32:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [10:32:52] <sd88g93> actually, it would be 3 hops, once to the thread, then once to the main thread, and then the forward device forwards them to the final socket for a send
[Wednesday 08 September 2010] [10:33:04] <cremes> only count sockets as hops
[Wednesday 08 September 2010] [10:33:38] <sd88g93> yeah, so 3 hops
[Wednesday 08 September 2010] [10:34:11] <cremes> why don’t you try using copy on the message received that you are sending to the inproc socket and see if that helps (shooting in the dark here)
[Wednesday 08 September 2010] [10:34:36] <sd88g93> yeah, i was trying move
[Wednesday 08 September 2010] [10:34:44] <sd88g93> but maybe copy would be better
[Wednesday 08 September 2010] [10:34:51] <cremes> so, XREQ(tcp) -> XREP(inproc) -> PUB(tcp)
[Wednesday 08 September 2010] [10:34:53] <sd88g93> i dont need to use it after relaying it
[Wednesday 08 September 2010] [10:34:56] <cremes> copy at the XREQ step
[Wednesday 08 September 2010] [10:35:32] <sd88g93> actually, i was using PUB for the inproc
[Wednesday 08 September 2010] [10:35:39] <cremes> the main thing to recall is that whenever you send through a socket, it calls close on the message you just sent
[Wednesday 08 September 2010] [10:35:49] <cremes> you relinquish control of the message to the library whenever you send anything
[Wednesday 08 September 2010] [10:36:13] <cremes> so it sounds to me like when you send to the inproc socket, the message may be getting closed on the send side
[Wednesday 08 September 2010] [10:36:21] <cremes> this is all kind of confusing :)
[Wednesday 08 September 2010] [10:36:33] <cremes> i recommend you ping mato or sustrik
[Wednesday 08 September 2010] [10:37:19] <sd88g93> in the worker thread, it is a PUB skt, and then in the main thread i have a sub skt, so i publish from the thread via inproc, and then it gets fwd’d to the main pub skt
[Wednesday 08 September 2010] [10:37:46] <sd88g93> oh ok
[Wednesday 08 September 2010] [10:37:57] <cremes> are you using a forwarder device or did you roll your own?
[Wednesday 08 September 2010] [10:38:05] <sd88g93> fwd device
[Wednesday 08 September 2010] [10:38:14] <cremes> are you using release 2.0.9?
[Wednesday 08 September 2010] [10:38:25] <sd88g93> yeah, i just updated to 2.0.9
[Wednesday 08 September 2010] [10:38:47] <cremes> ok... i think there was a bug in the forwarder for multi-part message handling that was fixed in 2.0.9
[Wednesday 08 September 2010] [10:39:03] <cremes> that could have been your problem, but if you updated then maybe not
[Wednesday 08 September 2010] [10:39:17] <sd88g93> at first i had my own queue to queue the messages for a publisher thread, that worked, except i’m worried about when i stress test it, if pasing messages allocated on the heap, is a problem when i free them
[Wednesday 08 September 2010] [10:39:46] <sd88g93> yeah, i noticed that in the change log when i updated
[Wednesday 08 September 2010] [10:39:52] <cremes> i’d have to see code to be sure but it doesn’t sound like a problem
[Wednesday 08 September 2010] [10:40:20] <cremes> sorry i couldn’t be of more help... if you figure it out please post a solution
[Wednesday 08 September 2010] [10:40:25] Quit gebi has left this server (Ping timeout: 252 seconds).
[Wednesday 08 September 2010] [10:40:46] <sd88g93> sure, no proble, i realize its still a young project, one of the ramifications
[Wednesday 08 September 2010] [10:41:25] <sd88g93> over all, ive noticed the documentation for zeromq getting much better and more concise as time goes on
[Wednesday 08 September 2010] [10:42:10] <sd88g93> i’ll try using message copy
[Wednesday 08 September 2010] [10:44:03] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [10:44:33] <cremes> great; you might also consider posting your question (with more detail) to the ML
[Wednesday 08 September 2010] [10:52:03] <pieterh> sd88g93: in fact I don’t know what cases need zmq_msg_move and zmq_msg_copy
[Wednesday 08 September 2010] [10:53:10] <sd88g93> ok, thanks, pieterh
[Wednesday 08 September 2010] [10:53:29] <pieterh> perhaps if you want to make abstraction layers that want to grab a copy of a message, work with it, and allow the caller to continue and possibly deallocate it
[Wednesday 08 September 2010] [10:53:48] <pieterh> you certainly do not need them for normal work afaics
[Wednesday 08 September 2010] [10:53:56] <cremes> pieterh: if you want to send the same data to multiple sockets, you need to copy it
[Wednesday 08 September 2010] [10:54:02] <sd88g93> yeah, the move/copy didnt work for my situtaion
[Wednesday 08 September 2010] [10:54:13] <cremes> otherwise your first send will deallocate it out from under you, right?
[Wednesday 08 September 2010] [10:55:03] <mato> cremes: send never deallocates anything out from under you. zmq_msg_t is reference counted.
[Wednesday 08 September 2010] [10:55:35] <cremes> mato: if refcount is 1 and you send the message, it gets deallocated, right?
[Wednesday 08 September 2010] [10:55:40] <sd88g93> oh, so a call to send increments the ref counter
[Wednesday 08 September 2010] [10:55:55] <cremes> i would think it decrements the counter
[Wednesday 08 September 2010] [10:56:03] <pieterh> ... this is all invisible... you send as often as you like and don’t even know there’s a reference counter
[Wednesday 08 September 2010] [10:56:11] <sd88g93> i mean, incrememnt at first, nd then decrement when its done wiht it
[Wednesday 08 September 2010] [10:56:32] <cremes> recv, refcount + 1, send, refcount - 1
[Wednesday 08 September 2010] [10:56:39] <cremes> when 0, dealloc
[Wednesday 08 September 2010] [10:56:44] <mato> cremes: zmq_send() does not touch the refcount
[Wednesday 08 September 2010] [10:56:58] <mato> cremes: you should always do zmq_msg_close () if you want the message to go away
[Wednesday 08 September 2010] [10:56:59] <cremes> doesn’t it call zmq_msg_close?
[Wednesday 08 September 2010] [10:57:12] <mato> not unless something changed; let me check 100%
[Wednesday 08 September 2010] [10:57:16] <cremes> ok
[Wednesday 08 September 2010] [10:57:26] <cremes> i thought i saw that when i reviewed the code a few weeks ago
[Wednesday 08 September 2010] [10:57:35] * pieterh is pretty sure send() does a copy and then a close
[Wednesday 08 September 2010] [10:57:52] <mato> pieterh: no way
[Wednesday 08 September 2010] [10:58:11] <mato> what all the perf tests and examples do is send, then immediately close
[Wednesday 08 September 2010] [10:58:37] <mato> unless i’m confused, check with sustrik to get it from the horses mouth
[Wednesday 08 September 2010] [10:58:41] <pieterh> yes, that’s the calling code
[Wednesday 08 September 2010] [10:58:47] <mato> but this is what i’ve always done in my code
[Wednesday 08 September 2010] [10:58:55] <pieterh> what does the send method do internally to keep the message alive
[Wednesday 08 September 2010] [10:59:02] <pieterh> while the caller does zmq_msg_close?
[Wednesday 08 September 2010] [10:59:08] <mato> ah, sorry
[Wednesday 08 September 2010] [10:59:19] <mato> yeah, send increases the refcount by 1 obviously
[Wednesday 08 September 2010] [10:59:27] <mato> but you get a refcount of 1 already when you create the message
[Wednesday 08 September 2010] [10:59:38] <pieterh> using copy, I assume... ?
[Wednesday 08 September 2010] [10:59:44] <mato> refcount
[Wednesday 08 September 2010] [10:59:49] <mato> what copy?
[Wednesday 08 September 2010] [10:59:59] <sd88g93> oh, so it increases by one, hten when its done sending , it decreases the ref count
[Wednesday 08 September 2010] [11:00:02] <pieterh> so directly modifying message property?
[Wednesday 08 September 2010] [11:00:12] <mato> some lock-free magic with atomic_counter AFAIK
[Wednesday 08 September 2010] [11:00:14] <mato> yes
[Wednesday 08 September 2010] [11:00:31] <mato> all the docs say “zmq_msg_t is opaque”, never touch it outside of the zmq_msg_* functions
[Wednesday 08 September 2010] [11:00:34] <mato> for precisely this reason
[Wednesday 08 September 2010] [11:00:38] <sd88g93> what about with VSM’s ?
[Wednesday 08 September 2010] [11:00:57] <mato> sd88g93: it’s up to 0MQ if it actually copies the content
[Wednesday 08 September 2010] [11:01:18] <pieterh> mato: this is what zmq_msg_copy does, increment refcount
[Wednesday 08 September 2010] [11:01:52] <sd88g93> so, if you have a VSM, and you send it, and then close it, if you re-init the struct, you wipe out what send would have ?
[Wednesday 08 September 2010] [11:02:15] <mato> pieterh: i’m not sure what it actually does internally, why do you need to know?
[Wednesday 08 September 2010] [11:02:19] <sd88g93> as far as i can tell, the VSM is right inside the struct
[Wednesday 08 September 2010] [11:02:23] <mato> pieterh: anyhow, if you need to know, ask sustrik
[Wednesday 08 September 2010] [11:02:27] <pieterh> sure
[Wednesday 08 September 2010] [11:02:35] <mato> sd88g93: ask sustrik on the mailing list, i’m not an expert on what happens internally
[Wednesday 08 September 2010] [11:02:37] <pieterh> the discussion here is “when do we use zmq_msg_copy/move”
[Wednesday 08 September 2010] [11:02:39] <mato> i just know that it works :)
[Wednesday 08 September 2010] [11:02:49] <pieterh> indeed...
[Wednesday 08 September 2010] [11:03:08] <pieterh> i’m just saying, applications do not normally mess with the refcounts, ever
[Wednesday 08 September 2010] [11:03:28] <sd88g93> mato, i’m just having a problem with relaying a message i get from a REP socket over PUB skt using inproc
[Wednesday 08 September 2010] [11:03:29] <pieterh> that’s internal, and i assume that send() calls copy() to grab a reference, and close() when it’s done, to release it
[Wednesday 08 September 2010] [11:03:29] <mato> well, unless you count calling zmq_msg_close() as “messing with the refcount”...
[Wednesday 08 September 2010] [11:03:50] <pieterh> well, sure, but that’s not the semantic
[Wednesday 08 September 2010] [11:03:59] <pieterh> the semantic is “I’m finished with it”, which translates to “decrement refcount”
[Wednesday 08 September 2010] [11:04:04] <mato> yup
[Wednesday 08 September 2010] [11:04:19] * pieterh doesn’t want people to start thinking they have to copy/move messages to send them
[Wednesday 08 September 2010] [11:04:31] <pieterh> sg88g93: the problem is elsewhere
[Wednesday 08 September 2010] [11:04:45] <mato> sd88g93: don’t forget zmq_recv() destroys existing message content, if any
[Wednesday 08 September 2010] [11:05:07] <pieterh> sd88g93, do you have a minimal example?
[Wednesday 08 September 2010] [11:05:28] <sd88g93> i have some code, but so far, i have been unablet to dupliate it in a minimal example,
[Wednesday 08 September 2010] [11:05:47] <mato> sd88g93: so e.g. if you’ve got a loop, doing zmq_msg_init() then zmq_recv(), what could be happening is you recv, then send that message, but then recv back into the same message so you could lose messages
[Wednesday 08 September 2010] [11:05:55] <sd88g93> i posted to the ML one example that i thought i duplicated it , but it didnt
[Wednesday 08 September 2010] [11:06:25] <pieterh> it copies messages from one socket to another, nothing bizarre
[Wednesday 08 September 2010] [11:06:41] <pieterh> if you’re losing messages, perhaps the sockets aren’t connected properly
[Wednesday 08 September 2010] [11:07:22] <mato> sd88g93/pieterh: AFAICT zmq_msg_copy()/zmq_msg_move() are for the case where the app for some reason re-uses message “objects” (i.e. the zmq_msg_t) with a different lifecycle than the message content
[Wednesday 08 September 2010] [11:07:39] <pieterh> right
[Wednesday 08 September 2010] [11:08:06] <mato> sd88g93: what language are you using?
[Wednesday 08 September 2010] [11:08:15] <sd88g93> the c bindings
[Wednesday 08 September 2010] [11:08:57] <pieterh> sd88g93, what is the problem you are actually seeing?
[Wednesday 08 September 2010] [11:09:36] <sd88g93> that’s a minimal case, that one works, but it mirrors what i’m doing in the larger project
[Wednesday 08 September 2010] [11:10:03] <pieterh> so publisher is not sending anything out
[Wednesday 08 September 2010] [11:10:04] <sd88g93> for some reason in the larger project the messages in the main thread arent being recieved via inproc by the subscr socket
[Wednesday 08 September 2010] [11:10:15] <sd88g93> yeah, the publisher isnt sending
[Wednesday 08 September 2010] [11:10:27] <sd88g93> ive isolated it to over the inproc that its failing
[Wednesday 08 September 2010] [11:10:40] <pieterh> try changing the transport to ipc
[Wednesday 08 September 2010] [11:10:55] <sd88g93> when i hcange it to one thread , and publish directly to the bound extreneral interface, it sends fine
[Wednesday 08 September 2010] [11:11:48] <pieterh> so it’s a problem with thread coordination, connections not happening in time
[Wednesday 08 September 2010] [11:11:50] <sd88g93> what’s the difference between inproc and ipc ?
[Wednesday 08 September 2010] [11:12:08] <pieterh> well, inproc requires that you bind first, connect second
[Wednesday 08 September 2010] [11:12:15] <pieterh> ipc like tcp lets you connect first, then bind
[Wednesday 08 September 2010] [11:12:24] <sd88g93> the connections seem to be happending, ive logged it and they seem to happen, the bound channels are made before the connect happens
[Wednesday 08 September 2010] [11:12:36] <pieterh> so with inproc if the subscribers connect before the publisher binds, it won’t work
[Wednesday 08 September 2010] [11:12:42] <pieterh> IMO connect will even return an error
[Wednesday 08 September 2010] [11:13:08] <sd88g93> no, it binds first, and then connects
[Wednesday 08 September 2010] [11:13:35] <sd88g93> that shouldnt be a problem, i think you are correct in that it returns an error, ive recieved that error when that happens
[Wednesday 08 September 2010] [11:13:39] <mato> sd88g93: this is what i meant with my comment about recv’ing into the same message:
http://pastebin.com/tLumsbia[Wednesday 08 September 2010] [11:13:46] <pieterh> yeah, this is your problem I think
[Wednesday 08 September 2010] [11:13:51] <mato> sd88g93: see the difference between the correct/incorrent case
[Wednesday 08 September 2010] [11:13:57] <mato> sd88g93: just in case that’s what you’re doing...
[Wednesday 08 September 2010] [11:14:02] <pieterh> you’re starting the publishers in child threads
[Wednesday 08 September 2010] [11:14:12] <pieterh> but immediately you connect the subscribers to those ports
[Wednesday 08 September 2010] [11:14:15] <pieterh> won’t work
[Wednesday 08 September 2010] [11:14:57] <pieterh> magic solution: do two loops, one to launch pub threads, second to prepare subscribers
[Wednesday 08 September 2010] [11:15:02] <pieterh> do sleep(1) in between two loops
[Wednesday 08 September 2010] [11:15:18] <pieterh> you owe me a beer if it works
[Wednesday 08 September 2010] [11:16:27] <pieterh> here is the text from the user guide:
[Wednesday 08 September 2010] [11:16:27] <pieterh> The {{inproc}} transport has a specific limitation compared to {{ipc}} and {{tcp}}: you must do bind before connect. This is something future versions of 0MQ may fix, but for today it has some impact on the way you use inproc sockets. In the example here, we carefully bind to each endpoint before connecting to it. If you connect first, and then bind, the connect will return an error, and if you ignore that error, the recv will block.
[Wednesday 08 September 2010] [11:17:29] <sd88g93> mato: i dont think that’ sthe case, because i re init the message at the beginning of the loop
[Wednesday 08 September 2010] [11:18:44] <sd88g93> pieterh: it seeems to be binding first before connect
[Wednesday 08 September 2010] [11:18:54] <pieterh> how do you know?
[Wednesday 08 September 2010] [11:19:03] <pieterh> your example does the bind after the connect
[Wednesday 08 September 2010] [11:19:03] <sd88g93> because i print it out
[Wednesday 08 September 2010] [11:19:09] <pieterh> means nothing
[Wednesday 08 September 2010] [11:19:19] <sd88g93> oh
[Wednesday 08 September 2010] [11:19:20] <pieterh> printing from multiple threads can do weird stuff
[Wednesday 08 September 2010] [11:19:35] <pieterh> follow the printf with fflush(stdout) to be sure
[Wednesday 08 September 2010] [11:19:41] <pieterh> sd88g93, next time you have an error, if 0MQ reports an error, that’s kind of the first thing you look at
[Wednesday 08 September 2010] [11:20:16] <pieterh> “I’m using inproc and my connect fails with an error... why?”
[Wednesday 08 September 2010] [11:20:33] <sd88g93> but the connect doesnt fail ,
[Wednesday 08 September 2010] [11:20:33] <
keffo__> is the log pub in 2.0.8?
[Wednesday 08 September 2010] [11:21:27] <pieterh>
keffo__, nope
[Wednesday 08 September 2010] [11:21:33] <pieterh> sd88g93, so what fails?
[Wednesday 08 September 2010] [11:21:46] <pieterh> and could you try the ‘magic fix’ i suggested, it’s a quick test
[Wednesday 08 September 2010] [11:21:57] <sd88g93> the program doesnt throw any errors, it just doesnt publish any of the messages
[Wednesday 08 September 2010] [11:22:08] <sd88g93> yes, i ‘ll tryt that test
[Wednesday 08 September 2010] [11:22:12] <pieterh> ” i think you are correct in that it returns an error, ive recieved that error when that happens”
[Wednesday 08 September 2010] [11:22:15] <pieterh> ?
[Wednesday 08 September 2010] [11:22:35] <sd88g93> sorry, i’m just reading over what everyone has written, its alot to digest
[Wednesday 08 September 2010] [11:22:44] <pieterh> NP :-)
[Wednesday 08 September 2010] [11:22:44] <sd88g93> i’ll try that fix now, thanks,
[Wednesday 08 September 2010] [11:22:58] <pieterh> did you try it with ipc:?
[Wednesday 08 September 2010] [11:23:24] <pieterh> if that works, and inproc does not, it’s most likely the bind/connect order
[Wednesday 08 September 2010] [11:37:17] <sd88g93> no, still nothing
[Wednesday 08 September 2010] [11:37:29] <sd88g93> i’ll try it with ipc , just to be sure
[Wednesday 08 September 2010] [11:37:40] <pieterh> ack
[Wednesday 08 September 2010] [11:39:19] <pieterh> there is some questionable stuff in your example BTW
[Wednesday 08 September 2010] [11:39:32] <pieterh> like threads modifying shared static variables
[Wednesday 08 September 2010] [11:40:39] <sd88g93> yeah, i dont do that n the main program
[Wednesday 08 September 2010] [11:41:34] <sd88g93> the shared static variable is just to get a number to form the pipe number back to the main program, in the actual example i pass it in through the parameter when creating the thread
[Wednesday 08 September 2010] [11:42:18] <pieterh> hang on, do you actually have any subscribers connecting to 127.0.0.1:4000?
[Wednesday 08 September 2010] [11:42:29] <sd88g93> yes
[Wednesday 08 September 2010] [11:43:14] <pieterh> well, what I would do is just break this up into little pieces
[Wednesday 08 September 2010] [11:43:17] <sd88g93> that example does seem to work, but the same scenario, doesnt play out in the real project
[Wednesday 08 September 2010] [11:43:30] <pieterh> ah, it works in the example?
[Wednesday 08 September 2010] [11:43:53] <sd88g93> yeah, just cant duplicate it , that example was just to give you an idea of what i am doing
[Wednesday 08 September 2010] [11:44:10] <pieterh> can’t debug an idea... :-/
[Wednesday 08 September 2010] [11:44:13] <sd88g93> sorry, didnt mean to confuse
[Wednesday 08 September 2010] [11:44:34] <sd88g93> yeah, trying to isolate where its coming from
[Wednesday 08 September 2010] [11:44:50] <pieterh> so you can write your own forwarding code
[Wednesday 08 September 2010] [11:44:55] <pieterh> it is trivial
[Wednesday 08 September 2010] [11:45:02] <pieterh> then you can debug that quite easily
[Wednesday 08 September 2010] [11:45:19] <sd88g93> but when i try a loop in the main program to read what’s coming from the subscriber socket via inproc, nothing comes from it
[Wednesday 08 September 2010] [11:45:31] <pieterh> your problem maybe is that by starting an in-built device you have a black box you can’t inspect
[Wednesday 08 September 2010] [11:45:58] <pieterh> so it’s not the publisher not sending, it’s the subscriber not receiving?
[Wednesday 08 September 2010] [11:46:28] <sd88g93> the publisher isnt making it to the outer publisher socket, nothings coming out
[Wednesday 08 September 2010] [11:46:48] <sd88g93> there’s not errors from the zmq_send function,
[Wednesday 08 September 2010] [11:47:30] <sd88g93> but when i monitor the socket with tshark, there’s nothing coming out
[Wednesday 08 September 2010] [11:47:37] <pieterh> sd88g93, sorry, I can’t really help you any more until/unless you can get a minimal test case that reproduces the problem
[Wednesday 08 September 2010] [11:47:47] Quit
keffo__ has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [11:47:59] <sd88g93> yeah, that’s the problem
[Wednesday 08 September 2010] [12:18:38] Quit
keffo__ has left this server (Ping timeout: 264 seconds).
[Wednesday 08 September 2010] [12:33:04] <cremes> sd88g93: i recommend instrumenting the forwarder device; just have it print what it receives and what it sends out again
[Wednesday 08 September 2010] [12:33:28] <cremes> if you are using multi-part messages with the topic in the first part, then perhaps it isn’t getting forwarded correctly and
[Wednesday 08 September 2010] [12:33:43] <cremes> your subscribers are dropping the messages because the topic doesn’t match
[Wednesday 08 September 2010] [12:43:22] <erickt> are any of the pyzmq devs around today?
[Wednesday 08 September 2010] [12:44:53] Quit
keffo__ has left this server (Ping timeout: 258 seconds).
[Wednesday 08 September 2010] [13:12:48] Join ModusPwnens has joined this channel (
~Slash@75.11.168.223).
[Wednesday 08 September 2010] [13:13:08] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [13:20:07] <ModusPwnens> hmm cremes, yesterday you said that if the performance tests were working correctly, remote_thr should be the first one to exist
[Wednesday 08 September 2010] [13:22:20] <ModusPwnens> exit*
[Wednesday 08 September 2010] [13:22:36] <cremes> right; the publisher should be done before the subscriber
[Wednesday 08 September 2010] [13:22:52] <cremes> but that doesn’t always happen on my machine so i’m a tad confused
[Wednesday 08 September 2010] [13:23:09] <cremes> unfortunately, i don’t have time to look at it right now... maybe someone else can take a peek
[Wednesday 08 September 2010] [13:23:10] <ModusPwnens> Yeah, what I am seeing is the remote continuing to run
[Wednesday 08 September 2010] [13:23:15] <ModusPwnens> until it times out or something
[Wednesday 08 September 2010] [13:24:37] <cremes> i recommend you add some print statements to the tests and run them with low iterations; perhaps something obvious will pop out
[Wednesday 08 September 2010] [13:29:03] <ModusPwnens> ok, will do
[Wednesday 08 September 2010] [13:30:08] Quit gebi has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [14:20:58] Quit
keffo__ has left this server (Ping timeout: 245 seconds).
[Wednesday 08 September 2010] [14:31:08] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Wednesday 08 September 2010] [14:35:55] Quit xla has left this server (Quit: leaving).
[Wednesday 08 September 2010] [14:52:27] Quit
keffo__ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [14:53:02] <mikko> hi
[Wednesday 08 September 2010] [14:55:21] <mikko> what happened with zmq_close semantics?
[Wednesday 08 September 2010] [15:02:30] Quit
keffo__ has left this server (Ping timeout: 272 seconds).
[Wednesday 08 September 2010] [15:21:20] <mato> mikko: the infrastructure work is all on ‘master’
[Wednesday 08 September 2010] [15:21:36] <mato> mikko: but i’ve not yet had time to test it properly due to “real” work :-(
[Wednesday 08 September 2010] [15:39:25] Quit
keffo__ has left this server (Read error: Connection reset by peer).
[Wednesday 08 September 2010] [16:20:18] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Wednesday 08 September 2010] [16:43:27] Quit
bennymack_ has left this server (Client Quit).
[Wednesday 08 September 2010] [16:45:29] <bennymack> hey everybody! So, i’m looking into 0MQ a bit and so I was strace() a simple example in perl and I noticed a lot of these 24byte send/recv calls on the pipes between the main thread and the 0MQ context thread
[Wednesday 08 September 2010] [16:45:54] <bennymack> is there something I can read about all that chatter that’s going on?
[Wednesday 08 September 2010] [16:46:07] <bennymack> is it the SPB?
[Wednesday 08 September 2010] [17:01:53] Quit GeekGod has left this server (Ping timeout: 276 seconds).
[Wednesday 08 September 2010] [17:01:53] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 08 September 2010] [17:03:33] Quit
GeekGod_ has left this server (Changing host).
[Wednesday 08 September 2010] [17:03:33] Nick
GeekGod_ is now known as GeekGod.
[Wednesday 08 September 2010] [17:09:37] <sd88g93> alhtough i dont think it gets into the low level stuff that youre talking about
[Wednesday 08 September 2010] [17:11:35] Quit psino has left this server (Quit: psino).
[Wednesday 08 September 2010] [17:12:45] <mato> bennymack: what you see is signalling between the application thread and zmq I/O thread, see zmq::signaler_t and read the source :-)
[Wednesday 08 September 2010] [17:12:46] <sd88g93> we were talking about it this morning, (or for you , i guess that would be afternoon)
[Wednesday 08 September 2010] [17:13:29] <sd88g93> oh, mato, you were here this morning, i managed to replicate the error i was having in this program :
http://pastebin.com/5Lsy47Cj[Wednesday 08 September 2010] [17:14:12] <mato> sd88g93: hmm, that’s long, do you have a 5-second summary?
[Wednesday 08 September 2010] [17:14:19] <sd88g93> ok,
[Wednesday 08 September 2010] [17:14:45] <sd88g93> the top program is fields requests from a client skt, and feeds them to different threads
[Wednesday 08 September 2010] [17:15:02] <sd88g93> it relays the multipart messages to a pub skt
[Wednesday 08 September 2010] [17:15:19] <sd88g93> (the second program is a client app to send it messages )
[Wednesday 08 September 2010] [17:15:52] <sd88g93> the problem is in the top program, it recives the messages fine , but doesnt relay the messages to the central publisher socket in the main thread
[Wednesday 08 September 2010] [17:16:33] <sd88g93> they should be easy to compile, just gcc -otestpub testpub.c -lzmq for the top program
[Wednesday 08 September 2010] [17:16:36] <mato> sd88g93: so did you find your problem?
[Wednesday 08 September 2010] [17:16:56] <sd88g93> i replicated the same problem in that code
[Wednesday 08 September 2010] [17:17:16] <sd88g93> its in the loop in the top program, (in the thread)
[Wednesday 08 September 2010] [17:17:19] <mato> sorry, it’s kind of late here ...
[Wednesday 08 September 2010] [17:17:25] <sd88g93> oh ok no problem
[Wednesday 08 September 2010] [17:17:26] <bennymack> ok thanks! I’ll take a look
[Wednesday 08 September 2010] [17:17:43] <sd88g93> I can post to the mailing list
[Wednesday 08 September 2010] [17:17:52] <mato> that might be best
[Wednesday 08 September 2010] [17:18:04] <sd88g93> yeah, still 5p.m. here
[Wednesday 08 September 2010] [17:18:09] <jond> sd88g93: is this the same problem as the code on the mail list. that needed a sleep after the pthread_create?
[Wednesday 08 September 2010] [17:18:33] <sd88g93> it might be, not sure which one youre thinking of
[Wednesday 08 September 2010] [17:18:40] <mato> jond: hi Jon... am in the middle of replying to your mail re Git
[Wednesday 08 September 2010] [17:19:16] <mato> jond: is your patmatch branch a private topic branch? i.e. noone pulling from it?
[Wednesday 08 September 2010] [17:19:41] <jond> mato: hi , thanks, also what does ‘Your branch is behind ‘origin/master’ by 86 commits, and can be fast-forwarded.’ mean?
[Wednesday 08 September 2010] [17:20:34] <jond> mato: yes, patmatch private topic branch which I intend to run format patch from and all be hunky dory
[Wednesday 08 September 2010] [17:20:36] <mato> jond: if it’s your local ‘master’ branch that means you can pull from it and you’re 86 commits behind...
[Wednesday 08 September 2010] [17:21:06] <mato> jond: the “can fast forward” stuff basically means you’ve not made any commits to your local master branch that would require a merge
[Wednesday 08 September 2010] [17:21:34] <jond> mato: yep local master; It;s been behind every since I did a hard reset that chuck posted, but i’m not clear how to get rid of it
[Wednesday 08 September 2010] [17:21:54] <mato> jond: git pull
[Wednesday 08 September 2010] [17:22:12] <jond> mato: this is after a git pull
[Wednesday 08 September 2010] [17:22:16] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Wednesday 08 September 2010] [17:23:00] <jond> mato: git pull says this now error: Untracked working tree file ‘src/connect_session.cpp’ would be overwritten by merge.
[Wednesday 08 September 2010] [17:23:22] <mato> jond: oh, right, you have changes you’ve not committed on local master
[Wednesday 08 September 2010] [17:23:28] <mato> jond: what do you want to do with those?
[Wednesday 08 September 2010] [17:24:29] <mato> jond: git status will show you what git thinks about your working tree state
[Wednesday 08 September 2010] [17:24:36] <jond> mato: there shouldnt be any, this all started when I pulled the 2.1 tree then reset to 2.0.8. I havent created those files or edited?
[Wednesday 08 September 2010] [17:24:55] <mato> jond: what does git status say?
[Wednesday 08 September 2010] [17:25:33] <mato> jond: alternatively, do you have any changes you care about in that repository?
[Wednesday 08 September 2010] [17:25:33] <jond> mato: git status gives me the 86 commits behind and some untracked files ; do i need to remove those?
[Wednesday 08 September 2010] [17:27:00] <jond> mato: thats fixed master
[Wednesday 08 September 2010] [17:27:10] <mato> ?
[Wednesday 08 September 2010] [17:27:33] <jond> i removed the ‘overwritten’ files
[Wednesday 08 September 2010] [17:27:48] <mato> you probably wanted git checkout – <path>
[Wednesday 08 September 2010] [17:28:03] <mato> does git status now say “working directory clean” ?
[Wednesday 08 September 2010] [17:29:34] <jond> mato: yes, after I removed some other note files I had left there
[Wednesday 08 September 2010] [17:29:55] <mato> jond: goodo, then you should be able to pull, and follow the instructions in my email
[Wednesday 08 September 2010] [17:30:10] <mato> jond: but if you want to be sure you don’t mess up just take a backup copy of your local repo
[Wednesday 08 September 2010] [17:30:20] <mato> jond: (cp -r zeromq2 zeromq2.bak or something)
[Wednesday 08 September 2010] [17:30:30] <jond> mato: will do , many thanks
[Wednesday 08 September 2010] [17:30:53] <mato> ok, i’m off... cyl
[Wednesday 08 September 2010] [17:31:01] <jond> mato: cyl
[Wednesday 08 September 2010] [17:31:09] <sd88g93> good night , mato
[Wednesday 08 September 2010] [17:31:15] <mato> nite, thx
[Wednesday 08 September 2010] [17:31:16] <sd88g93> ok, just posted to the ML
[Wednesday 08 September 2010] [17:36:42] Part sd88g93 has left this channel (“www.phash.org”).
[Wednesday 08 September 2010] [17:39:00] Quit andrewvc has left this server (Quit: andrewvc).
*** on Wed Sep 8 20:16:49 2010
[Wednesday 08 September 2010] [20:16:49] Topic The channel topic is “Welcome!”.
[Wednesday 08 September 2010] [20:16:55] Mode Channel modes: no messages from outside, topic protection
[Wednesday 08 September 2010] [20:16:55] Created This channel was created on 2010-02-05 10:44.
*** on Thu Sep 9 14:21:58 2010
[Thursday 09 September 2010] [14:21:58] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [14:22:04] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [14:22:04] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [14:30:21] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [14:30:23] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [14:30:24] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [14:30:24] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [14:30:31] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [14:30:31] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [14:30:48] <pieterh> PerfDave, sorry, i was kidding around
[Thursday 09 September 2010] [14:30:53] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [14:31:42] <pieterh> sd88g93, ok, test case is now clear, lemme think about it a sec
*** on Thu Sep 9 15:44:58 2010
[Thursday 09 September 2010] [15:44:58] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [15:45:04] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [15:45:04] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [15:45:13] <pieterh> sd88g93, what is ‘qskt’ for?
[Thursday 09 September 2010] [15:45:31] <sd88g93> that’s the rep socket, in the worker thread, that recieves the messages
[Thursday 09 September 2010] [15:45:35] <sd88g93> from the client
[Thursday 09 September 2010] [15:45:36] <pieterh> there is already a terrible lack of vowels here in Slovakia
[Thursday 09 September 2010] [15:46:06] <sd88g93> tough times, gotta conservee
[Thursday 09 September 2010] [15:46:22] <pieterh> that’s kind of like “who is that guy?” answer: “two legs, body, arms, head”
[Thursday 09 September 2010] [15:47:02] <pieterh> cause it does not in fact talk to the client at all
[Thursday 09 September 2010] [15:47:02] <sd88g93> couldnt come up with a more descriptive variable, sorry, i blame my lack of imagination
[Thursday 09 September 2010] [15:47:19] <sd88g93> it doesnt ? it does on mine
[Thursday 09 September 2010] [15:47:25] <pieterh> it talks to the main thread
[Thursday 09 September 2010] [15:47:26] <pieterh> via inproc
[Thursday 09 September 2010] [15:47:45] <sd88g93> yeah, then there’s a queue device to the main socket
[Thursday 09 September 2010] [15:47:47] <sd88g93> the client skt
[Thursday 09 September 2010] [15:48:11] <pieterh> why is the client not talking to the worker directly?
[Thursday 09 September 2010] [15:48:28] <sd88g93> if you have more than one thread
[Thursday 09 September 2010] [15:48:44] <sd88g93> use the device , at least that’s the scenario
[Thursday 09 September 2010] [15:48:55] <pieterh> well...
[Thursday 09 September 2010] [15:49:00] <pieterh> it
[Thursday 09 September 2010] [15:49:15] <pieterh> the device is really when you have an N-to-N case
[Thursday 09 September 2010] [15:49:23] <pieterh> and don’t want all the N’s to have to know about each other
[Thursday 09 September 2010] [15:49:43] <sd88g93> yeah, there’s just one client now, but there could be more
[Thursday 09 September 2010] [15:49:46] <pieterh> ah... you are in fact developing the MT server model...
[Thursday 09 September 2010] [15:49:54] <sd88g93> yeah
[Thursday 09 September 2010] [15:49:57] <pieterh> i hardly recognized it... :-/
[Thursday 09 September 2010] [15:49:59] <sd88g93> there could be more than one client
[Thursday 09 September 2010] [15:50:58] <sd88g93> well, a diffferent form of it, i guess
[Thursday 09 September 2010] [15:53:00] <sd88g93> zhelpers.h:25 i thought sprintf was “evil”, suppose to use snprintf ?
[Thursday 09 September 2010] [15:54:16] <keffo> pieterh, I’ve done the lru now!
[Thursday 09 September 2010] [15:54:34] <pieterh> keffo: wow!
[Thursday 09 September 2010] [15:54:41] <pieterh> sd88g93, this is what the worker loop becomes:
[Thursday 09 September 2010] [15:54:44] <pieterh> char
*command = s_recv (control);
[Thursday 09 September 2010] [15:54:44] <pieterh> char
*count = s_recv (control);
[Thursday 09 September 2010] [15:54:44] <pieterh> puts (“Worker got command: %sn”, command);
[Thursday 09 September 2010] [15:54:44] <pieterh> s_send (publisher, “msg.x|”);
[Thursday 09 September 2010] [15:54:47] <pieterh> s_sendmore (publisher, command);
[Thursday 09 September 2010] [15:54:47] <pieterh> s_send (publisher, count);
[Thursday 09 September 2010] [15:54:48] <pieterh> free (command);
[Thursday 09 September 2010] [15:54:50] <pieterh> free (count);
[Thursday 09 September 2010] [15:54:52] <pieterh> s_send (control, “OK”);
[Thursday 09 September 2010] [15:54:59] <pieterh> there is a bug, i see immediately...
[Thursday 09 September 2010] [15:55:03] <pieterh> in my code, not yours
[Thursday 09 September 2010] [15:55:24] <ModusPwnens> Wait a sec pieter, i was referring to something on the site
[Thursday 09 September 2010] [15:55:49] <sd88g93> oh, in s_send and s_sendmore you do init the message and close it each time,
[Thursday 09 September 2010] [15:56:16] <sd88g93> yeah, that’s what i thought, you were just referrring to coding style, eh ? not actuall instructions needed
[Thursday 09 September 2010] [15:56:24] <sd88g93> i get ya now, ok
[Thursday 09 September 2010] [15:56:50] <pieterh> ModusPwnens, even better, join the site and click ‘edit’
[Thursday 09 September 2010] [15:56:57] <pieterh> it’s a wiki :-)
[Thursday 09 September 2010] [15:57:07] <ModusPwnens> Oh okay, well that’s easy to fix then
[Thursday 09 September 2010] [15:57:09] <keffo> pieterh, it seems to be working very well.. I just need to post back the results now, but I’ll probably use a new socketlayer for that, with some persistance as well
[Thursday 09 September 2010] [15:57:25] <pieterh> hmm, nice
[Thursday 09 September 2010] [15:57:38] <pieterh> keffo: i still find working with envelopes to be too painful
[Thursday 09 September 2010] [15:57:52] <pieterh> but we’ll make abstractions for those later
[Thursday 09 September 2010] [15:58:47] <keffo> I have a Route class for that
[Thursday 09 September 2010] [15:59:02] <pieterh> yeah, right, you told me... nice
[Thursday 09 September 2010] [15:59:04] <keffo> which can construct itself from incoming, and also send it
[Thursday 09 September 2010] [15:59:07] <keffo> yeah
[Thursday 09 September 2010] [15:59:19] <pieterh> were the names in the routing section too weird?
[Thursday 09 September 2010] [15:59:27] <pieterh> i was trying to get mnemonics that worked
[Thursday 09 September 2010] [15:59:27] <keffo> where?
[Thursday 09 September 2010] [15:59:31] <pieterh> ch3
[Thursday 09 September 2010] [15:59:32] <keffo> mama,papa? =)
[Thursday 09 September 2010] [15:59:41] <pieterh> yeah
[Thursday 09 September 2010] [15:59:49] <jhawk28> the mama and papa were hilarious
[Thursday 09 September 2010] [15:59:52] <keffo> papa was a bit confusing, didnt really see the application to be honest, but I was mostly interested in the mama anyway :)
[Thursday 09 September 2010] [16:00:14] <pieterh> well, good, it’s hard to make this material even remotely funny
[Thursday 09 September 2010] [16:00:25] * pieterh still has to write a short piece on quick and clean exits
[Thursday 09 September 2010] [16:00:36] <jhawk28> we enjoyed it at work
[Thursday 09 September 2010] [16:00:37] <psino> pieterh: you should move some more reasoning about the naming in ch3 to the place where you introduce them
[Thursday 09 September 2010] [16:00:46] <keffo> pieterh, Another interesting sideffect of the mama topology you presented there is that there is no longer any immediate necessity for a heartbeat system
[Thursday 09 September 2010] [16:00:53] <pieterh> psino: did I get the order wrong?
[Thursday 09 September 2010] [16:01:07] <pieterh> yup, but you need heartbeats for another reason
[Thursday 09 September 2010] [16:01:26] <pieterh> i was going to write that today but got distracted by our friend sd88g93’s curious case here
[Thursday 09 September 2010] [16:01:36] <pieterh> also, ISP guys getting lost
[Thursday 09 September 2010] [16:01:38] <keffo> sure, but not in the same way as a topdown design
[Thursday 09 September 2010] [16:02:03] <psino> no, but when you say that REQ is a MAMA socket, making insistent demands, you might want to add something more that helps the reader remember the terminology
[Thursday 09 September 2010] [16:02:26] <pieterh> hmm, lemme think about that for a second...
[Thursday 09 September 2010] [16:02:52] <pieterh> i wrote that text very late on tuesday evening after a glass or two of wine... :-/
[Thursday 09 September 2010] [16:03:05] <psino> you describe the usage of mama pretty good in the “Router-to-Mama”-section
[Thursday 09 September 2010] [16:03:23] <psino> (personal opinion of course)
[Thursday 09 September 2010] [16:03:30] <pieterh> i think the character is based on my friend Fabio’ Italian mother
[Thursday 09 September 2010] [16:03:36] <pieterh> ok...
[Thursday 09 September 2010] [16:03:44] <pieterh> opinion is better than silence
[Thursday 09 September 2010] [16:04:33] <jhawk28> I was going through the swap and hwm parts of the user guide and I got an exception after the pub ran for a while (from yquery:108)
[Thursday 09 September 2010] [16:05:13] <pieterh> jhawk28, interesting!
[Thursday 09 September 2010] [16:05:23] <jhawk28> It died when I was using the connect, but not the bind
[Thursday 09 September 2010] [16:06:03] <jhawk28> not sure if it matters, but I was using the Java bindings
[Thursday 09 September 2010] [16:06:14] <pieterh> do you mean yqueue?
[Thursday 09 September 2010] [16:06:30] <jhawk28> yes, it was an assertion that failed
[Thursday 09 September 2010] [16:06:52] <jhawk28> going from memory - so it could be wrong
[Thursday 09 September 2010] [16:06:58] <pieterh> could you please log this on the issue tracker, along with the hardware setup you’re using?
[Thursday 09 September 2010] [16:07:12] <jhawk28> I think so
[Thursday 09 September 2010] [16:07:12] <pieterh> this should absolutely never never happen
[Thursday 09 September 2010] [16:07:27] <jhawk28> figured as much
[Thursday 09 September 2010] [16:07:30] <pieterh> there is indeed an assertion on yqueue.hpp:108
[Thursday 09 September 2010] [16:07:42] <pieterh> woa, hang on
[Thursday 09 September 2010] [16:07:49] <pieterh> it’s an out of memory condition
[Thursday 09 September 2010] [16:08:36] <jhawk28> hwm was set to 1 and swap was set to 25000000
[Thursday 09 September 2010] [16:09:27] <pieterh> you sure you didn’t reverse those? :-)
[Thursday 09 September 2010] [16:09:54] <pieterh> it’s worth logging, we’ll run that on some big boxes here and see if we can reproduce it
[Thursday 09 September 2010] [16:09:56] <jhawk28> :) was following the guide and then started playing around seeing what it would do
[Thursday 09 September 2010] [16:09:59] <pieterh> thanks
[Thursday 09 September 2010] [16:14:28] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [16:14:29] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [16:14:30] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [16:14:31] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [16:14:37] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [16:14:37] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [16:14:48] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [16:15:06] <daggertip> so
[Thursday 09 September 2010] [16:16:15] Part daggertip has left this channel.
[Thursday 09 September 2010] [16:17:16] * pieterh salutes the short-lived but nicely named daggertip
[Thursday 09 September 2010] [16:17:36] * cremes ha
[Thursday 09 September 2010] [16:23:10] <pieterh> sd88g93, you still there? I award you the “obfuscated 0MQ contest” award :-)
[Thursday 09 September 2010] [16:24:33] <pieterh> you have six sockets and two devices in your minimal test case
[Thursday 09 September 2010] [16:45:19] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [16:45:20] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [16:45:21] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [16:45:27] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [16:45:27] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [16:46:41] <sd88g93> i’m always up for an award
[Thursday 09 September 2010] [16:47:41] <sd88g93> think sustrik can have a looksie ?
[Thursday 09 September 2010] [16:54:15] <pieterh> sd88g93, lol, are you trolling me?
[Thursday 09 September 2010] [16:54:34] <sd88g93> trolling ?
[Thursday 09 September 2010] [16:54:36] <pieterh> i’ve stripped it right down and removing the queue device lets it work
[Thursday 09 September 2010] [16:54:39] <pieterh> :-)
[Thursday 09 September 2010] [16:54:56] <pieterh> gnash, i will find this... gnash
[Thursday 09 September 2010] [16:55:14] <sd88g93> how do you read the messages into the thread ?
[Thursday 09 September 2010] [16:55:28] <pieterh> well, connect worker directly to client
[Thursday 09 September 2010] [16:55:47] <pieterh> next time, develop such code step by step so you can test it gradually
[Thursday 09 September 2010] [16:56:03] <pieterh> and when you add a new piece if it does not work you know exactly what is questionable
[Thursday 09 September 2010] [16:56:07] <sd88g93> i’m suspicious that the messages are being altered in some way, like martin is slipping coded messages in there
[Thursday 09 September 2010] [16:56:18] <sd88g93> yeah, that’s what i did , but ....
[Thursday 09 September 2010] [16:56:24] <pieterh> i doubt it
[Thursday 09 September 2010] [16:56:29] <sd88g93> I realized zmq can do more
[Thursday 09 September 2010] [16:57:10] <sd88g93> I had a little different setup before, and then i put the forwarder back in , because i really didnt want to move zmq_msg_t ‘s around between threads
[Thursday 09 September 2010] [16:57:24] <pieterh> that is a messed up version but it shows you what it looks like
[Thursday 09 September 2010] [16:57:47] <pieterh> see how the queue thread creates its own sockets
[Thursday 09 September 2010] [16:58:06] <pieterh> not necessary to do that in main, kind of pointless and... in fact... illegal
[Thursday 09 September 2010] [16:58:15] <pieterh> you cannot create socket in thread A and use it in thread B
[Thursday 09 September 2010] [17:01:47] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [17:01:49] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [17:01:50] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:01:50] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:01:57] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:01:57] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:02:10] Quit
travlr_ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:05:13] <pieterh> ok, sd88g93 that’s the problem
[Thursday 09 September 2010] [17:05:30] <sd88g93> oh really
[Thursday 09 September 2010] [17:06:10] <pieterh> dude, you owe me a case of beer
[Thursday 09 September 2010] [17:06:36] <sd88g93> yes yes, i do hahaa
[Thursday 09 September 2010] [17:06:48] <pieterh> the publisher socket is not there, you can put it back of course
[Thursday 09 September 2010] [17:07:07] <pieterh> so problem was that your queue thread wasn’t working reliably
[Thursday 09 September 2010] [17:07:21] <pieterh> when you test code, don’t run it once, run it 10-100 times
[Thursday 09 September 2010] [17:07:31] <sd88g93> so you just create the skts for the device in that thread then,
[Thursday 09 September 2010] [17:07:35] <pieterh> threading issues are nasty like that
[Thursday 09 September 2010] [17:07:47] <pieterh> pls dnt sy skts t mks m brn hrt
[Thursday 09 September 2010] [17:08:01] <pieterh> socks if you really have to save on letters
[Thursday 09 September 2010] [17:08:12] <pieterh> but yes, you do
[Thursday 09 September 2010] [17:08:15] <sd88g93> what’s wrong with skts ?
[Thursday 09 September 2010] [17:08:32] <pieterh> what’s wrong with the letters ‘o’ and ‘e’?
[Thursday 09 September 2010] [17:08:33] <pieterh> :-)
[Thursday 09 September 2010] [17:08:52] <pieterh> seriously, if you remove essential letters from words you create extra parsing costs for the reader
[Thursday 09 September 2010] [17:08:59] <sd88g93> sorry, that’s just how i do variables, i tend to leave out vowels
[Thursday 09 September 2010] [17:09:08] <pieterh> english may look redundant but it’s been elegantly evolved for lazy readers
[Thursday 09 September 2010] [17:09:25] <pieterh> if you want a hint for style: make your code readable
[Thursday 09 September 2010] [17:09:37] <pieterh> please compare my versions with yours
[Thursday 09 September 2010] [17:09:58] <sd88g93> those zhelper.h are helpful, but unfortunately, i need it for binary data
[Thursday 09 September 2010] [17:10:01] <pieterh> you seriously cost me several hours just to untangle your code...
[Thursday 09 September 2010] [17:10:16] <pieterh> zhelpers.h is an example, modify/extend it for binary data
[Thursday 09 September 2010] [17:10:20] Quit psino has left this server (Quit: psino).
[Thursday 09 September 2010] [17:10:23] <pieterh> s_recv already works for binary data
[Thursday 09 September 2010] [17:10:30] <sd88g93> well, i was hoping for zero copy as well
[Thursday 09 September 2010] [17:10:32] <pieterh> add s_send_bin (socket, data, size) and it’s done
[Thursday 09 September 2010] [17:10:48] <sd88g93> ok,
[Thursday 09 September 2010] [17:10:58] <pieterh> just make sure to copy the data
[Thursday 09 September 2010] [17:11:04] <pieterh> otherwise you’ll get weirdness too
[Thursday 09 September 2010] [17:11:44] <pieterh> unless you are doing a lot of traffic (100K+ / second) or very large messages, zerocopy won’t be an advantage
[Thursday 09 September 2010] [17:11:52] <pieterh> but it will make your code harder to write and read and fix
[Thursday 09 September 2010] [17:11:53] <sd88g93> well, there’s no way to do zero copy though, when i can find a way to do zerocopy and those functions , then we’ve got something
[Thursday 09 September 2010] [17:12:09] <pieterh> do not optimize code that isn’t working
[Thursday 09 September 2010] [17:12:17] <pieterh> it is really a bad idea
[Thursday 09 September 2010] [17:12:31] <pieterh> actually I need to totally remove all reference to zerocopy from the user guide
[Thursday 09 September 2010] [17:12:37] <sd88g93> well, one message part can be pretty big
[Thursday 09 September 2010] [17:12:41] <pieterh> people should not even know it exists until chapter 4 or so...
[Thursday 09 September 2010] [17:12:47] <pieterh> doesn’t matter
[Thursday 09 September 2010] [17:12:55] <sd88g93> like 100k
[Thursday 09 September 2010] [17:13:02] <pieterh> doesn’t matter
[Thursday 09 September 2010] [17:13:03] <sd88g93> 100KB , easily
[Thursday 09 September 2010] [17:13:07] <pieterh> seriously, do the test yourself
[Thursday 09 September 2010] [17:13:21] <sd88g93> really ? it wont matter ?
[Thursday 09 September 2010] [17:13:36] <pieterh> for (namewithvowels = 0; nwv < 1000; nwv++) { do large copy here }
[Thursday 09 September 2010] [17:13:39] <sd88g93> hmm ok , i’ll have to think on that some
[Thursday 09 September 2010] [17:13:41] <pieterh> run, test, prove it to yourself
[Thursday 09 September 2010] [17:13:58] <pieterh> it matters if you are into microsecond latency issues and CPUs are running at 75% full
[Thursday 09 September 2010] [17:14:11] <pieterh> i’ll bet you that case of beer x 2 that’s not your case
[Thursday 09 September 2010] [17:14:41] <pieterh> premature optimization is one of the many roads to evil and it’s not even a particularly fun one
[Thursday 09 September 2010] [17:15:07] <gebi> zero-copy can also easily be more expensive than plain copy if not aligned on cache-lines and between different cpus
[Thursday 09 September 2010] [17:18:26] <pieterh> ok, folk, /me has to go sleepy byes
[Thursday 09 September 2010] [17:22:16] Nick Nickname already in use. Trying travlr.
[Thursday 09 September 2010] [17:22:17] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Thursday 09 September 2010] [17:22:17] Notice -NickServ- You are now identified for travlr.
[Thursday 09 September 2010] [17:22:18] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:22:19] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:22:25] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:22:25] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:22:57] Quit
travlr__ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:23:26] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [17:24:28] Quit sd88g93 has left this server (Quit: Ex-Chat).
[Thursday 09 September 2010] [17:31:42] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [17:31:44] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [17:31:44] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:31:45] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:31:48] Quit travlr has left this server (Ping timeout: 245 seconds).
[Thursday 09 September 2010] [17:31:52] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:31:52] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:55:12] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [17:55:14] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [17:55:15] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [17:55:15] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [17:55:22] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [17:55:22] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [17:56:18] Quit
travlr_ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [18:40:07] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [18:40:08] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [18:40:09] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [18:40:15] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [18:40:15] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [18:42:04] Quit kelvink has left this server (Quit: Leaving.).
[Thursday 09 September 2010] [18:59:09] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:04:37] Nick Nickname already in use. Trying travlr.
[Thursday 09 September 2010] [19:04:39] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Thursday 09 September 2010] [19:04:39] Notice -NickServ- You are now identified for travlr.
[Thursday 09 September 2010] [19:04:39] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:04:40] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:04:47] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:04:47] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:04:56] Quit
travlr__ has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:15:04] Nick Nickname already in use. Trying
travlr_.
[Thursday 09 September 2010] [19:15:06] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 09 September 2010] [19:15:06] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:15:07] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:15:14] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:15:14] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:15:34] Quit travlr has left this server (Ping timeout: 265 seconds).
[Thursday 09 September 2010] [19:25:31] Nick Nickname already in use. Trying
travlr__.
[Thursday 09 September 2010] [19:25:33] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 09 September 2010] [19:25:33] CTCP Received Version request from frigg.
[Thursday 09 September 2010] [19:25:34] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [19:25:40] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [19:25:41] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [19:26:23] Quit
travlr_ has left this server (Ping timeout: 276 seconds).
*** on Thu Sep 9 20:09:53 2010
[Thursday 09 September 2010] [20:09:53] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [20:09:59] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [20:09:59] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [20:18:18] Topic The channel topic is “Welcome!”.
[Thursday 09 September 2010] [20:18:24] Mode Channel modes: no messages from outside, topic protection
[Thursday 09 September 2010] [20:18:24] Created This channel was created on 2010-02-05 10:44.
[Thursday 09 September 2010] [20:19:11] Quit travlr has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 11 03:24:23 2010
[Saturday 11 September 2010] [03:24:23] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [03:24:29] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [03:24:29] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [03:31:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [03:53:07] Quit ivan has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [04:08:05] Quit rgl has left this server (Ping timeout: 272 seconds).
[Saturday 11 September 2010] [04:16:31] Join gebi has joined this channel (
~gebi@84.119.81.11).
[Saturday 11 September 2010] [05:22:59] Quit ivan has left this server (Ping timeout: 276 seconds).
*** on Sat Sep 11 06:07:38 2010
[Saturday 11 September 2010] [06:07:38] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [06:07:44] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [06:07:44] Created This channel was created on 2010-02-05 10:44.
*** on Sat Sep 11 06:32:47 2010
[Saturday 11 September 2010] [06:32:47] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [06:32:53] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [06:32:53] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [07:44:15] <icy> I can connect a socket to many endpoints, what is the reason for not allowing one to disconnect from an endpoint again?
[Saturday 11 September 2010] [08:00:29] <icy> also why do I have to loop over all sockets to see which ones got events after a zmq_poll(), I can’t see how this would scale
[Saturday 11 September 2010] [08:03:36] <guido_g> that’s how poll works
[Saturday 11 September 2010] [08:08:57] <icy> well that kinda nullifies a benefit that for example epoll() gives you
[Saturday 11 September 2010] [08:09:05] <icy> where you don’t have to loop around searching
[Saturday 11 September 2010] [08:11:16] <guido_g> a) somewhere you have to go through the list of fds
[Saturday 11 September 2010] [08:12:11] <guido_g> b) mq sockets aggregate os level sockets (one mq socket -> many os level sockets), so you have only a few mq sockets
[Saturday 11 September 2010] [08:12:37] <guido_g> c) mq uses epoll or whatever is available on the given platform
[Saturday 11 September 2010] [08:12:42] <icy> yea but due to not being able to disconnect from one endpoint, I might end up needing many zmq sockets
[Saturday 11 September 2010] [08:13:07] <guido_g> i don’t dare to ask why...
[Saturday 11 September 2010] [08:13:08] <icy> it uses epoll but makes me loop over all sockets, epoll does not require me to do that
[Saturday 11 September 2010] [08:13:45] <guido_g> epoll is not available on all platforms mq is running on
[Saturday 11 September 2010] [08:13:49] <icy> I know
[Saturday 11 September 2010] [08:13:52] <guido_g> so?
[Saturday 11 September 2010] [08:14:02] <icy> but many provide smarter facilities than poll()
[Saturday 11 September 2010] [08:14:44] <icy> so you throw away a lot of benefits for a lowest common denominator
[Saturday 11 September 2010] [08:15:03] <guido_g> so you think it would make sense to have mq behave differently on differnt platforms?
[Saturday 11 September 2010] [08:15:17] <icy> that is not what I am saying
[Saturday 11 September 2010] [08:15:21] <guido_g> it is
[Saturday 11 September 2010] [08:15:27] <icy> nope
[Saturday 11 September 2010] [08:15:55] <guido_g> because if you use the _best_ way on a platform, you also have the platform specific semantics
[Saturday 11 September 2010] [08:16:15] <icy> you can still abstract it in a general way
[Saturday 11 September 2010] [08:16:24] <icy> see all event libraries like libev or libevent
[Saturday 11 September 2010] [08:16:56] <gebi> icy: just for the record, epoll is NOT faster than poll in some situations, quite frankly it is 20% or more slower than poll in some situations
[Saturday 11 September 2010] [08:17:11] <icy> I know, for small sets poll() is nice
[Saturday 11 September 2010] [08:17:35] <guido_g> may be these abstraction would cost a lot on some platforms, so why punish people not running you favorite platform?
[Saturday 11 September 2010] [08:18:05] <icy> they do not cost any considerable amount
[Saturday 11 September 2010] [08:18:46] <guido_g> evidence by allegation?
[Saturday 11 September 2010] [08:19:01] <icy> what evidence do you provide?
[Saturday 11 September 2010] [08:19:48] <guido_g> mq is built that way, and works
[Saturday 11 September 2010] [08:21:32] <icy> well all I’m saying is I cannot see how it can scale to many zmq sockets because of all the looping. so clearly there must be a reason for that and that would that you would not need many zqm sockets as each can connect to many transport endpoints
[Saturday 11 September 2010] [08:22:07] <icy> which brings me to my initial question why you can’t disconnect from one again
[Saturday 11 September 2010] [08:23:15] <icy> I might be missing something in how you are supposed to handle many volatile connections and am ready for enlightenment :)
[Saturday 11 September 2010] [08:25:11] <guido_g> what kind of connections?
[Saturday 11 September 2010] [08:25:32] <guido_g> you don’t see connects at the mq level
[Saturday 11 September 2010] [08:28:26] <icy> hm I was thinking of a socket which connects to many IPs over tcp but I guess I just need to flip it around and make it the server and not client
[Saturday 11 September 2010] [08:29:25] <guido_g> even then you would have only one mq socket
[Saturday 11 September 2010] [08:29:33] <pieter_hintjens> gebi: yes, i recall Zed Shaw’s analysis, quite fun
[Saturday 11 September 2010] [08:30:03] <pieter_hintjens> icy: you’re IMO trying to apply existing TCP patterns to 0MQ
[Saturday 11 September 2010] [08:30:03] <icy> yea but if the IPs are dynamic, the zmq socket would just get more and more endpoints attached to it
[Saturday 11 September 2010] [08:30:17] <icy> pieter_hintjens: quite possible, I am used to that :)
[Saturday 11 September 2010] [08:30:55] <guido_g> if you connect more and more sockets to a remote socket, then yes, the number of connections is growing
[Saturday 11 September 2010] [08:30:57] <pieter_hintjens> icy: you need to think of 0MQ sockets as mini applications not just APIs
[Saturday 11 September 2010] [08:31:08] <pieter_hintjens> they are intelligent and do things for you
[Saturday 11 September 2010] [08:31:24] <pieter_hintjens> learning this is the key to using 0MQ right and getting it to scale and be useful for you
[Saturday 11 September 2010] [08:31:54] <pieter_hintjens> there are some generic rules that apply to all 0MQ sockets
[Saturday 11 September 2010] [08:32:02] <pieter_hintjens> and then there are specifics to each type, how they behave
[Saturday 11 September 2010] [08:32:12] <pieter_hintjens> generically, client vs. server is important to get right
[Saturday 11 September 2010] [08:32:35] <pieter_hintjens> stable nodes act better as servers (doing the bind), dynamic nodes as clients (doing the connect)
[Saturday 11 September 2010] [08:32:36] <icy> yup makes sense, I think it will just take a bit of time to get the old patterns out of my head
[Saturday 11 September 2010] [08:32:39] <pieter_hintjens> yup
[Saturday 11 September 2010] [08:32:50] <pieter_hintjens> if it’s any help, we all go through this
[Saturday 11 September 2010] [08:32:59] <pieter_hintjens> did you read the user guide and try the examples?
[Saturday 11 September 2010] [08:33:05] <icy> yes
[Saturday 11 September 2010] [08:33:16] <pieter_hintjens> did you just read it, or try the code?
[Saturday 11 September 2010] [08:33:36] <icy> I tried not all but some
[Saturday 11 September 2010] [08:33:38] <icy> and read all
[Saturday 11 September 2010] [08:33:44] <pieter_hintjens> well, it takes time too
[Saturday 11 September 2010] [08:33:58] <pieter_hintjens> several days of slow reprogramming
[Saturday 11 September 2010] [08:34:02] <guido_g> it needs time to sink in
[Saturday 11 September 2010] [08:34:15] <guido_g> mq can’t be groked in a day, imho
[Saturday 11 September 2010] [08:34:46] <pieter_hintjens> if you never did networking programming you’d have less trouble unlearning old patterns
[Saturday 11 September 2010] [08:34:51] <guido_g> the things that makes it so good is what makes it so hard :)
[Saturday 11 September 2010] [08:34:53] <pieter_hintjens> but you’d still have to learn the new stuff
[Saturday 11 September 2010] [08:35:09] <guido_g> but who would that be?
[Saturday 11 September 2010] [08:35:11] <pieter_hintjens> icy: briefly, you trust 0MQ sockets to do a lot for you
[Saturday 11 September 2010] [08:35:34] <pieter_hintjens> there is a lot less of the command-and-control you need with traditional sockets
[Saturday 11 September 2010] [08:36:32] <icy> well I understand how stuff works, not very complicated after all. I’m trying now to understand why the api is how it is. e.g. normally event libs let you specify callbacks that get called upon events (hence I was so surprised by the looping needed after zmq_poll)
[Saturday 11 September 2010] [08:36:51] <pieter_hintjens> icy: the api aims to look like plain boring sockets
[Saturday 11 September 2010] [08:37:05] <pieter_hintjens> the poll stuff is not very nice
[Saturday 11 September 2010] [08:37:13] <pieter_hintjens> and we’ll probably make it nicer for 3.0
[Saturday 11 September 2010] [08:37:19] <icy> fair enough
[Saturday 11 September 2010] [08:37:37] <pieter_hintjens> there is a moral when you make new technology
[Saturday 11 September 2010] [08:37:43] <pieter_hintjens> if it looks new, people won’t adopt it
[Saturday 11 September 2010] [08:38:16] <guido_g> hmmm
[Saturday 11 September 2010] [08:38:42] <pieter_hintjens> people = except pioneers who have a fetish for new technology just on principle
[Saturday 11 September 2010] [08:38:54] <pieter_hintjens> and who will abandon it for the next new thing without hesitation
[Saturday 11 September 2010] [08:39:26] <guido_g> i hmmed because the last 30 years have seen a lot of new things
[Saturday 11 September 2010] [08:39:40] <pieter_hintjens> I read a lovely quote from the inventor of liquid salt nuclear reactors
[Saturday 11 September 2010] [08:39:54] <pieter_hintjens> sure... i’m making a horrid generalization
[Saturday 11 September 2010] [08:39:59] <guido_g> ack
[Saturday 11 September 2010] [08:39:59] <icy> I’m not so sure if this can be applied so easily
[Saturday 11 September 2010] [08:40:02] <guido_g> go on :)
[Saturday 11 September 2010] [08:40:11] <pieter_hintjens> “Perhaps the moral to be drawn is that a technology that differs too much from an existing technology has not one hurdle to overcome–to demostrate its feasibility–but another even greater one–to convince influential individuals and organizations who are intellectually and emotionally attached to a different technology that they should adopt the new path”
[Saturday 11 September 2010] [08:40:29] <guido_g> icy: it’s a trade-off between usability and speed, mostly
[Saturday 11 September 2010] [08:40:30] <icy> if it was true, no radically things would have been adopted in the past but there have been plenty :)
[Saturday 11 September 2010] [08:40:50] <pieter_hintjens> ah, (a) there are always exceptions to a general rule and (b) there are ways to appear old yet introduce newness
[Saturday 11 September 2010] [08:40:56] <guido_g> i’ve seen a messaging lib w/ callbacks, and it wasn’t fun at all
[Saturday 11 September 2010] [08:41:20] <icy> in my opinion anything providing a considerable advantage over something else has good chances of getting adopted
[Saturday 11 September 2010] [08:41:36] <icy> provided it does not have disadvantages that outweight the advantages of course :P
[Saturday 11 September 2010] [08:41:44] <guido_g> you remember the video-tape thingy?
[Saturday 11 September 2010] [08:41:46] <pieter_hintjens> this is the engineer point of view but it ignores people emotional attachment to stuff they know
[Saturday 11 September 2010] [08:41:52] <pieter_hintjens>
*people’s
[Saturday 11 September 2010] [08:41:57] <pieter_hintjens>
*peoples’
[Saturday 11 September 2010] [08:42:06] <pieter_hintjens> sms text messages
[Saturday 11 September 2010] [08:42:10] <pieter_hintjens> email
[Saturday 11 September 2010] [08:42:10] <icy> guido_g: what about fire, electricity etc? :)
[Saturday 11 September 2010] [08:42:22] <pieter_hintjens> icy: fire has callbacks?
[Saturday 11 September 2010] [08:42:30] <icy> there are examples for both sides
[Saturday 11 September 2010] [08:42:35] <icy> pieter_hintjens: uhm.. ;)
[Saturday 11 September 2010] [08:42:40] <pieter_hintjens> :-)
[Saturday 11 September 2010] [08:43:03] <guido_g> icy: but it doesn’t work that way
[Saturday 11 September 2010] [08:43:10] <pieter_hintjens> anyhow, this is why 0MQ very carefully emulates a poll function everyone hates but totally trusts and understands
[Saturday 11 September 2010] [08:44:12] <icy> guido_g: just curious, why was that lib no fun?
[Saturday 11 September 2010] [08:44:25] <guido_g> icy: wllm
[Saturday 11 September 2010] [08:44:39] <icy> pieter_hintjens: I totally see your point
[Saturday 11 September 2010] [08:45:02] <icy> wllm?
[Saturday 11 September 2010] [08:45:09] <pieter_hintjens> icy: once there is a significant weight behind 0MQ (and we’re getting there) we can be more ambitious about our own abstractions
[Saturday 11 September 2010] [08:45:41] <guido_g> icy: from a small 3 letter company (HAL) :)
[Saturday 11 September 2010] [08:46:12] <icy> pieter_hintjens: wouldn’t it be hard to make a radical api change once you got a considerable amount of applications using it? as a counter argument
[Saturday 11 September 2010] [08:46:17] <icy> guido_g: will take a look, thx
[Saturday 11 September 2010] [08:46:21] <pieter_hintjens> icy: it’s about layers
[Saturday 11 September 2010] [08:46:28] <guido_g> icy: no need to, really :)
[Saturday 11 September 2010] [08:46:31] <pieter_hintjens> we’d never make radical api changes
[Saturday 11 September 2010] [08:46:41] <pieter_hintjens> just add layers on top
[Saturday 11 September 2010] [08:47:14] <pieter_hintjens> e.g. event driven frameworks that actually call the poll API
[Saturday 11 September 2010] [08:47:32] <pieter_hintjens> though I suspect poll API itself will be redesigned before the 0MQ is totally stabilized
[Saturday 11 September 2010] [08:47:40] <guido_g> so the developer can pick his poison depending on the use-case etc.
[Saturday 11 September 2010] [08:52:10] <pieter_hintjens> yeah
[Saturday 11 September 2010] [08:52:25] <icy> for the non blocking part I’d argue it makes it even easier given the atomicity of messages
[Saturday 11 September 2010] [08:53:50] <guido_g> we had a lot of trouble w/ these callbacks
[Saturday 11 September 2010] [08:54:32] <guido_g> one problem is that you are still in the library
[Saturday 11 September 2010] [08:54:45] <guido_g> which leads to more or less well known constraints
[Saturday 11 September 2010] [08:55:06] <pieter_hintjens> threading issues etc.
[Saturday 11 September 2010] [08:55:07] <guido_g> i wouldn’t go that route again
[Saturday 11 September 2010] [08:55:15] <pieter_hintjens> the reactor pattern looks more sane
[Saturday 11 September 2010] [08:55:17] <guido_g> pieter_hintjens: oh yes!
[Saturday 11 September 2010] [08:55:27] <guido_g> to both
[Saturday 11 September 2010] [08:55:36] <pieter_hintjens> what i’d love to make is a Libero/0MQ at some stage
[Saturday 11 September 2010] [08:56:04] <guido_g> sorry, what would that be?
[Saturday 11 September 2010] [08:56:08] <pieter_hintjens> well, something like that, anyhow: state machines where events are 0MQ messages
[Saturday 11 September 2010] [08:56:42] <pieter_hintjens> you turn the loop inside out, the loop calls your code and feeds it incoming messages as events
[Saturday 11 September 2010] [08:56:49] <pieter_hintjens> and your code goes through states
[Saturday 11 September 2010] [08:56:57] <guido_g> makes sense
[Saturday 11 September 2010] [08:57:13] <pieter_hintjens> it’s a pattern that can be terribly useful but it can be hard to grok as well
[Saturday 11 September 2010] [08:57:33] <pieter_hintjens> only worth doing for more complex 0MQ apps but I suspect we’re heading that way
[Saturday 11 September 2010] [08:57:46] <pieter_hintjens> devices that manage 3+ sockets
[Saturday 11 September 2010] [08:57:46] <guido_g> sure
[Saturday 11 September 2010] [08:59:52] <icy> yes this is what I was refering to. you give a socket a function to call upon receiving a message. this way you don’t need to care about searching which sockets got a message after a poll
[Saturday 11 September 2010] [09:00:12] <icy> zmq would do it for you and can therefor use the faster implementation possible for a given platform
[Saturday 11 September 2010] [09:00:29] <pieter_hintjens> icy: you can trivially build this on top
[Saturday 11 September 2010] [09:00:36] <pieter_hintjens> perhaps a fun thing to make
[Saturday 11 September 2010] [09:00:59] <guido_g> icy: as i said, it wasn’t nice to use in a real world app
[Saturday 11 September 2010] [09:01:08] <pieter_hintjens> i can see several problems with it, though
[Saturday 11 September 2010] [09:01:32] <guido_g> plus the problems comming from the product itself
[Saturday 11 September 2010] [09:01:33] <pieter_hintjens> passing control to a function means you can’t e.g. work with multiple sockets at once
[Saturday 11 September 2010] [09:01:45] <pieter_hintjens> so you can’t do stuff like devices that read from one, write to the other
[Saturday 11 September 2010] [09:01:58] <pieter_hintjens> or you need to create and pass around increasingly complex contexts
[Saturday 11 September 2010] [09:01:58] <guido_g> you can’t even do a proper error handling :)
[Saturday 11 September 2010] [09:02:19] <guido_g> which then slow you down even more
[Saturday 11 September 2010] [09:02:24] <pieter_hintjens> but it could be fun to experiment with
[Saturday 11 September 2010] [09:02:48] <pieter_hintjens> icy: take a look at the various reactors people like Matt Weinstein and Chuck Remes have made
[Saturday 11 September 2010] [09:04:08] <icy> of course you can do proper error handling. maybe your experience was because of a poor product, not the methodology itself guido_g
[Saturday 11 September 2010] [09:04:32] <icy> but yea to handle multiple sockets you need to pass around also some additional data
[Saturday 11 September 2010] [09:05:05] <guido_g> callbacks are generally not the first choice, i did X11 and Motif and learned to avoid them
[Saturday 11 September 2010] [09:05:16] <guido_g> then wllm, smame thing
[Saturday 11 September 2010] [09:05:21] <guido_g>
*same
[Saturday 11 September 2010] [09:05:51] <guido_g> if you want it, you should do it in the app
[Saturday 11 September 2010] [09:06:31] <guido_g> libs or frameworks can’t know enough of the use-cases to provide a proper solution for every user
[Saturday 11 September 2010] [09:06:58] <guido_g> this is why frameworks tend to become so big and bloated
[Saturday 11 September 2010] [09:09:01] <pieter_hintjens> icy: what i’ve found with 0MQ is that theory is pretty useless :-)
[Saturday 11 September 2010] [09:09:12] <pieter_hintjens> get your hands dirty with code and it becomes rapidly clear
[Saturday 11 September 2010] [09:09:24] <pieter_hintjens> happily what you learn is the same no matter what language you use
[Saturday 11 September 2010] [11:11:05] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 11 September 2010] [11:58:53] Quit keffo has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [13:08:35] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [13:14:50] Quit gebi has left this server (Read error: Operation timed out).
[Saturday 11 September 2010] [13:24:13] <keffo> um, push/pull is not pub/sub, or did i confuse things?
[Saturday 11 September 2010] [13:25:01] <keffo> ah right
[Saturday 11 September 2010] [13:37:30] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [13:58:22] Join old_sound has joined this channel (
~mrhyde@222.44.41.33).
[Saturday 11 September 2010] [13:59:01] Part old_sound has left this channel.
[Saturday 11 September 2010] [14:01:25] Quit xla has left this server (Ping timeout: 260 seconds).
[Saturday 11 September 2010] [14:11:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Saturday 11 September 2010] [14:14:38] Quit rbraley has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [15:04:18] <iFire> I mean is it feasible
*** on Sat Sep 11 16:30:13 2010
[Saturday 11 September 2010] [16:30:13] Topic The channel topic is “Welcome!”.
[Saturday 11 September 2010] [16:30:19] Mode Channel modes: no messages from outside, topic protection
[Saturday 11 September 2010] [16:30:19] Created This channel was created on 2010-02-05 10:44.
[Saturday 11 September 2010] [17:22:47] Quit rgl has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [18:04:34] <psino> is there any way to tell a socket to stop connecting to an endpoint, or stop listening on an endpoint?
[Saturday 11 September 2010] [18:19:59] <psino> and is there a way of gracefully shutting down a worker process that uses an incoming PULL socket?
[Saturday 11 September 2010] [18:37:44] Quit keffo has left this server (Ping timeout: 276 seconds).
[Saturday 11 September 2010] [18:53:43] Quit troutwine has left this server (Remote host closed the connection).
[Saturday 11 September 2010] [19:06:05] Quit plq has left this server (Ping timeout: 258 seconds).
[Saturday 11 September 2010] [23:22:58] Quit rgl has left this server (Ping timeout: 252 seconds).
[Saturday 11 September 2010] [23:33:09] Quit kooroo has left this server (Quit: HydraIRC ->
http://www.hydrairc.com <-).
[Sunday 12 September 2010] [00:22:53] Quit Samy has left this server (
*.net
*.split).
[Sunday 12 September 2010] [00:22:54] Quit mivert has left this server (
*.net
*.split).
[Sunday 12 September 2010] [01:01:42] Quit Samy has left this server (Excess Flood).
[Sunday 12 September 2010] [01:01:43] Quit rbraley has left this server (Excess Flood).
[Sunday 12 September 2010] [01:02:03] Quit soren has left this server (Remote host closed the connection).
[Sunday 12 September 2010] [01:18:59] Quit Tasser has left this server (
*.net
*.split).
[Sunday 12 September 2010] [01:42:38] Quit beppu has left this server (Ping timeout: 245 seconds).
[Sunday 12 September 2010] [01:49:53] Quit xla has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [02:12:18] Quit jsimmons has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [03:36:03] <pieter_hintjens> iFire: I’m going to make an example of that in Ch3
[Sunday 12 September 2010] [03:36:06] <pieter_hintjens> of the guide
*** on Sun Sep 12 03:46:31 2010
[Sunday 12 September 2010] [03:46:31] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [03:46:37] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [03:46:37] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [03:49:06] <iFire> pieter_hintjens chapater ch3 where
[Sunday 12 September 2010] [03:50:47] <iFire> where will the chapter 3 be pieter_hintjens?
[Sunday 12 September 2010] [03:51:20] <guido_g> user guid
[Sunday 12 September 2010] [03:51:22] <guido_g> +e
[Sunday 12 September 2010] [07:53:47] Quit psino has left this server (Ping timeout: 255 seconds).
[Sunday 12 September 2010] [08:23:28] Quit rgl has left this server (Ping timeout: 255 seconds).
[Sunday 12 September 2010] [09:12:21] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [10:14:10] Quit psino has left this server (Quit: psino).
[Sunday 12 September 2010] [10:45:27] Quit omarkj has left this server (Quit: omarkj).
[Sunday 12 September 2010] [11:44:45] Quit
benoitc_ has left this server (Changing host).
[Sunday 12 September 2010] [11:45:00] Nick
benoitc_ is now known as benoitc.
[Sunday 12 September 2010] [12:58:50] Quit jsimmons has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [13:22:40] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [13:27:37] Quit
keffo_ has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [13:47:41] <mankins> messages can be of arbitrary length, right?
[Sunday 12 September 2010] [13:56:35] <icy> they have to fit in ram
[Sunday 12 September 2010] [14:21:18] Quit gebi has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [14:31:50] Quit
gebi_ has left this server (Ping timeout: 264 seconds).
[Sunday 12 September 2010] [14:49:05] <mankins> i’m having some issue with messages getting corrupted on the wire.
[Sunday 12 September 2010] [14:49:29] <mankins> trying to isolate it.
[Sunday 12 September 2010] [14:49:54] <mankins> there are known issues with 2.0.9, are there? (I checked the bug list, and nothing jumped out)
[Sunday 12 September 2010] [14:52:35] Quit gebi has left this server (Ping timeout: 252 seconds).
[Sunday 12 September 2010] [15:11:32] Quit gebi has left this server (Ping timeout: 265 seconds).
[Sunday 12 September 2010] [15:17:39] <mankins> hrm, maybe it was the perl serializer.
[Sunday 12 September 2010] [15:18:14] <mankins> i was using send/recv but changed to send_as/recv and it seems to be better. <shrug>
[Sunday 12 September 2010] [15:34:31] Quit gebi has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [15:44:10] Quit rgl has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [15:53:05] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [15:54:10] Quit
keffo__ has left this server (Read error: Connection reset by peer).
[Sunday 12 September 2010] [16:00:02] Quit gebi has left this server (Ping timeout: 264 seconds).
[Sunday 12 September 2010] [16:03:28] Quit mankins has left this server (Quit: mankins).
[Sunday 12 September 2010] [16:19:41] Quit rgl has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [16:32:54] Quit
soren_ has left this server (Changing host).
[Sunday 12 September 2010] [16:33:44] Nick
soren_ is now known as soren.
[Sunday 12 September 2010] [16:45:56] Quit plq has left this server (Quit: Leaving).
[Sunday 12 September 2010] [17:11:53] Quit gebi has left this server (Ping timeout: 276 seconds).
[Sunday 12 September 2010] [17:39:22] Part bobs has left this channel.
[Sunday 12 September 2010] [18:28:18] <kenkeiter> Looking for some input on a decentralized pub-sub/dequeue model with zeromq
[Sunday 12 September 2010] [18:30:23] <kenkeiter> need to be able to publish messages in a specific category, subscribe to that category and (hopefully) pull jobs from it in a load-balanced way.
[Sunday 12 September 2010] [18:30:28] <kenkeiter> Any suggestions?
[Sunday 12 September 2010] [19:26:18] Quit
keffo__ has left this server (Ping timeout: 272 seconds).
[Sunday 12 September 2010] [19:26:22] <sleeperbot> Hey guys, wondering if I could get some idea on what’s going on with my installation of zmq
[Sunday 12 September 2010] [19:26:39] <sleeperbot> been using zeromq and my CPU usage shot up x100. I’m doing Upstream/Downstream, using the node.js wrapper.
[Sunday 12 September 2010] [19:26:46] <sleeperbot> Leaving connections open on both ends to send and receive
[Sunday 12 September 2010] [19:32:15] <kenkeiter> Don’t know a tremendous amount about zmq’s internals, but my suggestion would be to verify that you have the correct number of zmq threads spawned.
[Sunday 12 September 2010] [19:32:49] <kenkeiter> If you’re respawning for each connection, they may also not be released properly..
[Sunday 12 September 2010] [19:32:57] <kenkeiter> Which could pose another problem.
*** on Sun Sep 12 19:37:10 2010
[Sunday 12 September 2010] [19:37:10] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [19:37:16] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [19:37:16] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [19:43:52] Quit andrewvc has left this server (Ping timeout: 272 seconds).
[Sunday 12 September 2010] [19:53:57] <sleeperbot> Thanks Ken, i’ll check
[Sunday 12 September 2010] [19:54:21] <sleeperbot> I’m not respawning so it could be the node.js driver
[Sunday 12 September 2010] [19:59:18] Quit GeekGod has left this server (Quit: GeekGod).
[Sunday 12 September 2010] [20:00:10] <kenkeiter> sleeperbot: it was worth a try.. did you verify that you’re only running one or two messaging threads?
[Sunday 12 September 2010] [20:00:13] <andrewvc> I’ve been curious as to how stable the node driver is
[Sunday 12 September 2010] [20:00:20] <kenkeiter> :/
[Sunday 12 September 2010] [20:03:14] <sleeperbot> I’m looking up how to do that
[Sunday 12 September 2010] [20:03:52] <sleeperbot> do you know what I can type in the command line to bring that info up?
[Sunday 12 September 2010] [20:04:03] <kenkeiter> sleeperbot: which platform?
[Sunday 12 September 2010] [20:04:25] <sleeperbot> unix
[Sunday 12 September 2010] [20:04:30] <sleeperbot> ubuntu karmic
[Sunday 12 September 2010] [20:07:43] <kenkeiter> htop might work.. haven’t done it under
*ni
[Sunday 12 September 2010] [20:14:13] <sleeperbot> I see 3 versions of my node.js stream and web servers
[Sunday 12 September 2010] [20:14:27] <sleeperbot> don’t see anything related to zmq
[Sunday 12 September 2010] [20:15:25] <sleeperbot> killed the extraneous processes, will check if anything changed in cpu usage
*** on Sun Sep 12 20:17:38 2010
[Sunday 12 September 2010] [20:17:38] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [20:17:43] Part andrewvc has left this channel.
[Sunday 12 September 2010] [20:17:44] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [20:17:44] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [20:21:22] Quit bennymack has left this server (Ping timeout: 252 seconds).
[Sunday 12 September 2010] [20:33:24] Quit psino has left this server (Quit: psino).
[Sunday 12 September 2010] [20:39:02] Quit jsimmons has left this server (Ping timeout: 264 seconds).
*** on Sun Sep 12 21:09:03 2010
[Sunday 12 September 2010] [21:09:03] Topic The channel topic is “Welcome!”.
[Sunday 12 September 2010] [21:09:09] Mode Channel modes: no messages from outside, topic protection
[Sunday 12 September 2010] [21:09:09] Created This channel was created on 2010-02-05 10:44.
[Sunday 12 September 2010] [22:57:18] Part pieter_hintjens has left this channel.
[Sunday 12 September 2010] [23:25:32] Quit kenkeiter has left this server (Quit: kenkeiter).
[Sunday 12 September 2010] [23:34:09] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Sunday 12 September 2010] [23:34:24] <andrewvc> I assume that XREQ/XREP sockets apply backpressure in the same manner as PUSH/PULL and REQ/REP yes?
[Sunday 12 September 2010] [23:59:40] Quit sleeperbot has left this server (Ping timeout: 252 seconds).
[Monday 13 September 2010] [00:15:34] Quit xla has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [00:44:16] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [00:56:55] Quit bbigras has left this server (Ping timeout: 252 seconds).
[Monday 13 September 2010] [01:30:06] Quit guido_g has left this server (Quit: Boom...).
[Monday 13 September 2010] [01:53:45] Quit
keffo__ has left this server (Read error: Connection reset by peer).
*** on Mon Sep 13 02:24:03 2010
[Monday 13 September 2010] [02:24:03] Topic The channel topic is “Welcome!”.
[Monday 13 September 2010] [02:24:09] Mode Channel modes: no messages from outside, topic protection
[Monday 13 September 2010] [02:24:09] Created This channel was created on 2010-02-05 10:44.
[Monday 13 September 2010] [02:36:59] Quit rgl has left this server (Ping timeout: 255 seconds).
[Monday 13 September 2010] [02:47:16] Quit kenkeiter has left this server (Quit: kenkeiter).
[Monday 13 September 2010] [03:18:13] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [04:37:41] Join plq has joined this channel (
~plq@78.189.30.218).
[Monday 13 September 2010] [05:08:18] Part keffo has left this channel (“Leaving”).
[Monday 13 September 2010] [07:09:02] Quit omarkj has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [07:09:03] Nick
omarkj_ is now known as omarkj.
[Monday 13 September 2010] [07:28:04] <CIA-20> zeromq2: 03Martin Lucina 07master * rbe159b6 10/ src/pipe.cpp : zmq::writer_t: Add missing test for swap - http://bit.ly/aGN4bM
[Monday 13 September 2010] [07:29:19] <icy> sustrik: hi, is there any paper on the algorithm used for the lock-free queue?
[Monday 13 September 2010] [07:30:51] <sustrik> icy: there’s a very old article here:
[Monday 13 September 2010] [07:30:59] <sustrik> lot of it doesn’t apply any more
[Monday 13 September 2010] [07:31:42] <sustrik> this is what still applies: “+ Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> - Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> FoldUnfold
[Monday 13 September 2010] [07:31:42] <sustrik> Table of Contents
[Monday 13 September 2010] [07:31:42] <sustrik> Introduction
[Monday 13 September 2010] [07:31:44] <sustrik> Design
[Monday 13 September 2010] [07:31:46] <sustrik> Performance
[Monday 13 September 2010] [07:31:48] <sustrik> Configuration
[Monday 13 September 2010] [07:31:50] <sustrik> Performance with polling
[Monday 13 September 2010] [07:31:52] <sustrik> Performance without polling
[Monday 13 September 2010] [07:31:54] <sustrik> Conclusion
[Monday 13 September 2010] [07:31:56] <sustrik> HISTORICAL WHITEPAPER
[Monday 13 September 2010] [07:31:58] <sustrik> Introduction
[Monday 13 September 2010] [07:32:00] <sustrik> Y-suite is a set of components designed for ultra-efficient passing of messages between threads within a process. Y-suite is somehow similar to local sockets, however, it is much faster.
[Monday 13 September 2010] [07:32:03] <sustrik> In version 0.1 of MQ lightweight messaging kernel, the only y-suite component available is ypipe, a lock-free and wait-free implementation of a queue. In version 0.2 ypollset is added to allow thread to interchange messages with several other threads at the same time (similar to POSIX poll function). Component known as semaphore in version 0.1 is renamed to ysemaphore in version 0.2 to mark that it belongs to y-suite. Same way, spipe i
[Monday 13 September 2010] [07:32:08] <sustrik> s renamed to ysocketpair.
[Monday 13 September 2010] [07:32:10] <sustrik> Design
[Monday 13 September 2010] [07:32:14] <sustrik> The basic means of transferring message between threads is ypipe. Messages are passed through a pipe in the standard write and read manner. Once the reader has no more messages to read from the pipe, it notifies the sender using passive synchronization and goes asleep. Passive synchronization means that the other thread is not notified directly using some kind of async signal, rather it will be notified once it tries to write the next me
[Monday 13 September 2010] [07:32:19] <sustrik> ssage to the pipe. When this happens, writer becomes aware that reader is already asleep or at least going asleep at the moment. It knows that there is new message available, so it wakes the reader up using active synchronization, i.e. actively sending wake-up event to the other thread. Active synchronisation is not provided by ypipe itself, rather by other y-suite components, to be discussed bellow. Usage of ypipe is depicted on the fol
[Monday 13 September 2010] [07:32:24] <sustrik> lowing sequence diagram:”
[Monday 13 September 2010] [07:32:26] <sustrik> yuck
[Monday 13 September 2010] [07:32:28] <sustrik> sorry
[Monday 13 September 2010] [07:33:01] <sustrik> too much text, but the last paragraph is relevant
[Monday 13 September 2010] [07:33:36] <sustrik> also see the diagram that follows the text above
[Monday 13 September 2010] [07:34:23] <ekidd> Good morning! ZeroMQ is a really nice library.
[Monday 13 September 2010] [07:35:08] <ekidd> If I’m using REQ/REP messaging with multiple servers, what happens if one server is asked to handle an unusually long-running request?
[Monday 13 September 2010] [07:36:29] <ekidd> Do the clients just route requests to one of the available servers? Or do they continue to send requests to the busy server?
[Monday 13 September 2010] [07:36:30] <sustrik> ekidd: if you set high watermark, it’s queue gets eventually full and subsequent requests will be dispatches to other servers
[Monday 13 September 2010] [07:36:59] <icy> sustrik: yea I’ve read that, I guess because it is single-reader single-write, it does not suffer from the ABA problem?
[Monday 13 September 2010] [07:37:38] <ekidd> sustrik: Ah, OK. The useful high watermark in my case is very small: The servers are inherently single-threaded workers with long-running jobs. I want to keep them loaded.
[Monday 13 September 2010] [07:37:45] <sustrik> icy: what’s ABA?
[Monday 13 September 2010] [07:38:02] <ekidd> I do, however, have lots of clients and servers.
[Monday 13 September 2010] [07:39:11] <icy> sustrik: it’s one of the main problems that lock-free queues have to overcome
[Monday 13 September 2010] [07:39:23] <guido_g> it takes an unusually long time to complete the request
[Monday 13 September 2010] [07:40:01] <ekidd> icy: My clients and servers are different machines, so I don’t think the lock-free stuff is relevant. But I might be confused.
[Monday 13 September 2010] [07:40:29] <sustrik> ekidd: that’s a different conversion going on :)
[Monday 13 September 2010] [07:40:40] <ekidd> Ah, OK. I was confused. :-)
[Monday 13 September 2010] [07:41:02] <guido_g> ekidd: did you see that req/req is locked to the send/recv order?
[Monday 13 September 2010] [07:41:02] <sustrik> ekidd: there’s no such thing in 0MQ as explicit ack
[Monday 13 September 2010] [07:41:27] <sustrik> so there’s no way for it to work in lock-step fashion
[Monday 13 September 2010] [07:41:32] <ekidd> guido_g: Yeah, that works for me.
[Monday 13 September 2010] [07:42:32] <sustrik> icy: it’s basically a two step process
[Monday 13 September 2010] [07:42:32] <guido_g> same for the rep side (the server)
[Monday 13 September 2010] [07:42:41] <ekidd> I basically have a farm of Windows workers that take 0.1 to (say) 60 seconds to process a job, and idle time costs money. There’s one worker per server.
[Monday 13 September 2010] [07:42:46] <guido_g> see the user guide for lots of examples and ideas
[Monday 13 September 2010] [07:43:12] <sustrik> icy: while there are messages to be read the synchronisation is done simply by moving a pointer in the linked list in atomic manner
[Monday 13 September 2010] [07:43:32] <sustrik> icy: when there are no messages to be read, the pointer becomes NULL
[Monday 13 September 2010] [07:43:32] <ekidd> I don’t mind locking the order of responses the way req/rep does: I’m talking to expensive, single-threaded Windows libraries in any case.
[Monday 13 September 2010] [07:44:01] <icy> sustrik: understood so far and it seems it does not suffer from aba, just was curious if there was real proof of the correctness of the algorithm
[Monday 13 September 2010] [07:44:05] <sustrik> icy: reader goes asleep and standard inter-thread mechanism (socketpair) is used to wake it up
[Monday 13 September 2010] [07:44:16] <ekidd> But I want to maximize utilization of those expensive libraries.
[Monday 13 September 2010] [07:44:23] <sustrik> icy: no
[Monday 13 September 2010] [07:44:29] <sustrik> want to prove it?
[Monday 13 September 2010] [07:44:46] <icy> that would take more time than I have probably :)
[Monday 13 September 2010] [07:45:08] <ekidd> As long as zeromq clients respect the individual server’s high water marks and route requests to another worker, everything will work fine.
[Monday 13 September 2010] [07:46:09] <ekidd> I’m going to write some tests (of course). I just wanted to know whether I was even trying something sane. :-)
[Monday 13 September 2010] [07:48:01] <ekidd> Many thanks for your advice, folks!
[Monday 13 September 2010] [07:54:15] <ekidd> guido_g: Excellent. It definitely has the right semantics. I’ll still need to find out whether it does the right thing, performance-wise, with large messages and queues that are often at their high water marks.
[Monday 13 September 2010] [08:37:13] Quit rgl has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [08:39:00] Part gebi has left this channel.
[Monday 13 September 2010] [08:51:31] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [09:27:32] Quit plq has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [09:33:25] Quit guido_g has left this server (Quit: Boom...).
[Monday 13 September 2010] [09:43:13] Quit rbraley has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [09:44:53] <cremes> while writing some specs for my bindings this weekend i came across a few issues with SWAP, RECOVERY_IVL and RATE
[Monday 13 September 2010] [09:45:08] <cremes> all 3 of those take signed 64-bit integers for input
[Monday 13 September 2010] [09:45:32] <cremes> they also do not return an error when passed a negative number even though that doesn’t make any sense
[Monday 13 September 2010] [09:45:56] <cremes> should the library return an error for negative numbers or should my bindings take care of that issue?
[Monday 13 September 2010] [09:49:52] Join plq has joined this channel (
~plq@78.189.30.218).
[Monday 13 September 2010] [09:55:33] Quit jsimmons has left this server (Ping timeout: 245 seconds).
[Monday 13 September 2010] [09:58:11] <ptrb> I start the server, it sits at zmq_recv(), great; i run the client, it runs fine and exits, but the server never receives anything. ideas?
[Monday 13 September 2010] [10:00:05] <pieterh> cremes: i think all the setsockopt types need to be reviewed for 3.0
[Monday 13 September 2010] [10:00:41] <cremes> pieterh: ok; so should i open bugs for those against the 2.1.x branch?
[Monday 13 September 2010] [10:00:42] <pieterh> but certainly if they are signed get a negative value that should return EINVAL
[Monday 13 September 2010] [10:00:51] <pieterh> yup, even 2.0.x IMO
[Monday 13 September 2010] [10:00:57] <cremes> ok, i’ll do that now
[Monday 13 September 2010] [10:01:21] <pieterh> ptrb: looking at it...
[Monday 13 September 2010] [10:02:01] <ptrb> pieterh: thx; I’m guessing there’s some setup step I’ve overlooked
[Monday 13 September 2010] [10:02:23] <pieterh> ptrb: try ‘ps’
[Monday 13 September 2010] [10:02:31] <pieterh> imo you have a second copy of the server running
[Monday 13 September 2010] [10:02:52] <pieterh> (though it would assert then...)
[Monday 13 September 2010] [10:03:11] <pieterh> sorry, forget I said that plz
[Monday 13 September 2010] [10:03:11] <ptrb> hmm, no, but maybe something else is sitting on 5001, let me try changing that
[Monday 13 September 2010] [10:03:22] <pieterh> ptrb: client writes a message and then closes & exits
[Monday 13 September 2010] [10:03:28] <pieterh> two things: (a) it should wait for a reply
[Monday 13 September 2010] [10:03:37] <pieterh> (b) if it does not want to wait, it can’t exit immediately
[Monday 13 September 2010] [10:03:44] <pieterh> you need to read the users guide
[Monday 13 September 2010] [10:04:00] <ptrb> I have.
[Monday 13 September 2010] [10:04:10] <pieterh> 0mq/2.0.x loses data if you close the socket while there is data in flight
[Monday 13 September 2010] [10:04:11] <cremes> ptrb: are you starting the server first?
[Monday 13 September 2010] [10:04:16] <ptrb> Of course.
[Monday 13 September 2010] [10:04:23] <pieterh> send/close is not going to work
[Monday 13 September 2010] [10:04:28] <pieterh> send/recv/close is ok
[Monday 13 September 2010] [10:04:32] <pieterh> send/sleep/close is ok
[Monday 13 September 2010] [10:04:32] <ptrb> OK, so, do I need to recv() in the clie... k
[Monday 13 September 2010] [10:04:34] <cremes> ah yes, that’s right
[Monday 13 September 2010] [10:04:43] <cremes> do a sleep before exiting
[Monday 13 September 2010] [10:04:45] <ptrb> even if I don’t post anything back explicitly?
[Monday 13 September 2010] [10:04:47] <pieterh> prtb: either a recv
[Monday 13 September 2010] [10:04:59] Nick
bbigras_ is now known as bbigras.
[Monday 13 September 2010] [10:05:07] <pieterh> prtb if you’re using REQ and REP sockets, you should be doing send/recv and recv/send
[Monday 13 September 2010] [10:05:20] <pieterh> if you want to just send 1 message as such use PUSH/PULL
[Monday 13 September 2010] [10:05:39] <pieterh> it’s not a biggie
[Monday 13 September 2010] [10:05:52] <pieterh> the problem here is not giving the client process time to send its data
[Monday 13 September 2010] [10:05:53] <ptrb> I’m doing something vaguely RPC-ish, so I guess if I want to represent a void blah(); I still have to send something back
[Monday 13 September 2010] [10:06:06] <pieterh> or else use XREQ/XREP
[Monday 13 September 2010] [10:06:09] <ptrb> yeah it makes sense, sure. thanks. i guess it’s just not explicit anywhere in the docs (afaict)
[Monday 13 September 2010] [10:06:18] <pieterh> rtfug... :-)
[Monday 13 September 2010] [10:06:22] <pieterh> it is explicit in there
[Monday 13 September 2010] [10:06:31] <ptrb> i have; if you want to point me to the sentence in question I’m happy to be made a fool
[Monday 13 September 2010] [10:06:46] * pieterh goes off to search for ‘sleep’...
[Monday 13 September 2010] [10:07:11] <pieterh> Note that we do sleep (1); before exiting the ventilator. This is a hack that gets around MQ/2.0’s design, which discards messages that have not yet been sent, if you exit the program too soon. If you are using MQ/2.1 you can remove this sleep statement.
[Monday 13 September 2010] [10:07:43] <ptrb> eh.
[Monday 13 September 2010] [10:08:19] <pieterh> it’s the first example that has this problem, so I explain it there
[Monday 13 September 2010] [10:08:29] <pieterh> the hello world client waits for an answer
[Monday 13 September 2010] [10:08:38] <pieterh> and the pubsub example never exits
[Monday 13 September 2010] [10:09:01] <pieterh> maybe i should put it in bold...
[Monday 13 September 2010] [10:09:13] <pieterh> and repeat this, it’s a common fault
[Monday 13 September 2010] [10:11:29] <ptrb> if you’re willing to take some constructive criticism about the documentation, i’d say that while example-based docs are great, when I have a specific problem (like this) I find there isn’t really a way I can find a solution; there’s no idioms or implementation details or whatever to search through (as far as I’ve found)
[Monday 13 September 2010] [10:13:56] <ptrb> but!! but but, thank you :)
[Monday 13 September 2010] [10:19:21] <ptrb> hmm, recv on the client side hangs... is there not some zmq_flush or something I can call?
[Monday 13 September 2010] [10:20:33] <guido_g> no
[Monday 13 September 2010] [10:20:40] <ptrb> poop :|
[Monday 13 September 2010] [10:21:06] <guido_g> pardon?
[Monday 13 September 2010] [10:21:19] <ptrb> that was an expression of mild disappointment
[Monday 13 September 2010] [10:31:20] Quit plq has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [10:40:40] Quit mato has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [11:04:45] Quit rbraley has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [11:16:08] Quit mivert has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [11:16:51] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [11:35:55] <pieterh> ptrb: still there?
[Monday 13 September 2010] [11:36:04] <pieterh> sorry, was in a meeting
[Monday 13 September 2010] [11:36:19] <ptrb> yeah sure
[Monday 13 September 2010] [11:36:54] <pieterh> making a problem driven section in the guide would be good
[Monday 13 September 2010] [11:37:01] <pieterh> did you find out why your client hangs?
[Monday 13 September 2010] [11:37:29] <ptrb> No, I just threw a sleep in there and moved on to bigger, even more problematic things :)
[Monday 13 September 2010] [11:37:41] <ptrb> a problem-driven section would be good, but it’ll never be comprehensive
[Monday 13 September 2010] [11:38:09] <ptrb> FWIW I think a good documentation model would be ZeroC’s ICE, which has a really comprehensive .pdf
[Monday 13 September 2010] [11:38:16] <pieterh> “did not get a message” is a pretty classic stumbling block
[Monday 13 September 2010] [11:38:23] <ptrb> yeah, fair
[Monday 13 September 2010] [11:38:33] <pieterh> i’ll write a flowchart
[Monday 13 September 2010] [11:40:41] <ptrb> now, i’m working on an implementation based on the multithreaded code in the user guide, and i’m getting infinite size-0 messages on the server side after sending one legitimate message from a client
[Monday 13 September 2010] [11:40:48] <ptrb> ever hear of something like this?
[Monday 13 September 2010] [11:42:33] <ptrb> sorry, based on the multithreaded server in the introduction doc
[Monday 13 September 2010] [11:43:41] <cremes> ptrb: i’ve never seen that... you say your code is “based on” the example; it’s always a good idea to start from code that you know works and modify from there
[Monday 13 September 2010] [11:43:49] <cremes> sounds like your mods broke it
[Monday 13 September 2010] [11:44:08] <cremes> the easiest way to find the failure is to revert back to the original “good” code and slowly modify it to your specifications
[Monday 13 September 2010] [11:44:27] <ptrb> yeah. i know. i’m trying to drop the server into an existing process to provide a zmq “layer”, so there’s not really any way to iterate my way to where I am now.
[Monday 13 September 2010] [11:45:07] <ptrb> i guess i can try taking out some functionality.
[Monday 13 September 2010] [11:45:20] <cremes> did you change the code that sends 0mq messages?
[Monday 13 September 2010] [11:48:35] <ptrb> yes; in ways i initially thought were inconsequential, but i suppose i’m in an assumption-revalidating mood :)
[Monday 13 September 2010] [11:49:15] <ptrb> as a meta-comment, it’s really great you guys are hanging out on irc to help folks; zmq is a great project and this is a great resource.
[Monday 13 September 2010] [11:50:54] <ptrb> aha! so, if i zmq_recv(), get a message, and don’t zmq_send() something in the server, subsequent zmq_recv()s have the effect of not blocking
[Monday 13 September 2010] [11:51:09] <ptrb> ...which seems quite strange to me
[Monday 13 September 2010] [11:51:57] <cremes> ptrb: what kind of socket are you using on this server side?
[Monday 13 September 2010] [11:53:06] <cremes> because that behavior doesn’t sound right; the zmq_recv() call is returning an error, right?
[Monday 13 September 2010] [11:54:04] <ptrb> yeah, it returns -1 EAGAIN
[Monday 13 September 2010] [11:54:11] <ptrb> i believe it’s EAGAIN, at least.
[Monday 13 September 2010] [11:56:31] <ptrb> the topology is the multithread server example in the intro doc: public tcp XREP endpoint, managed by one thread running zmq_device(ZMQ_QUEUE, ...), forwarding via XREQ to an inproc endpoint, being consumed by worker threads binding to REP
[Monday 13 September 2010] [12:02:56] <cremes> can you provide a code pastie?
[Monday 13 September 2010] [12:03:14] <ptrb> it won’t be complete, but sure, one sec...
[Monday 13 September 2010] [12:03:38] <cremes> it doesn’t need to be complete... i want to see the code that sets up the socket and calls recv on it
[Monday 13 September 2010] [12:04:03] <ptrb> the worker thread ultimately responsible for processing the recv, right?
[Monday 13 September 2010] [12:04:39] <cremes> whatever code is returning -1 EAGAIN
[Monday 13 September 2010] [12:06:16] Quit xla has left this server (Quit: leaving).
[Monday 13 September 2010] [12:08:10] <cremes> ptrb: in your DEBUG statement, also print out the value of zmq_strerror()
[Monday 13 September 2010] [12:08:27] <cremes> i need more information to figure this out
[Monday 13 September 2010] [12:09:15] Quit xla has left this server (Client Quit).
[Monday 13 September 2010] [12:09:18] <ptrb> Operation cannot be accomplished in current state
[Monday 13 September 2010] [12:10:02] <cremes> ah, then there we have it; with a REP socket you can’t call recv again until you have subsequently called send
[Monday 13 September 2010] [12:10:16] <cremes> it needs that recv/send/recv pattern because it maintains a small internal state machine
[Monday 13 September 2010] [12:10:23] <ptrb> oh, interesting
[Monday 13 September 2010] [12:10:24] <cremes> that’s the whole point of the REQ/REP socket pattern
[Monday 13 September 2010] [12:10:24] <ptrb> ok
[Monday 13 September 2010] [12:10:34] <ptrb> see, this is useful! this should be on a website somewhere :)
[Monday 13 September 2010] [12:10:40] <cremes> the worker is supposed to respond when it is done, right?
[Monday 13 September 2010] [12:10:50] <ptrb> well, my thought is that it may optionally respond
[Monday 13 September 2010] [12:10:51] <cremes> it is for sure
[Monday 13 September 2010] [12:10:58] <ptrb> but if it has to respond, that’s fine too
[Monday 13 September 2010] [12:11:07] <cremes> if you want it to be optional, use XREP sockets
[Monday 13 September 2010] [12:11:25] <cremes> that kind of socket does not enforce the recv/send/recv pattern
[Monday 13 September 2010] [12:11:45] <ptrb> I suspect there is more to XREP than simply dropping that enforcement, though
[Monday 13 September 2010] [12:12:53] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [12:13:51] <cremes> ptrb: not really; REP sockets are built on top of the XREP socket
[Monday 13 September 2010] [12:14:00] <ptrb> hmm, interesting
[Monday 13 September 2010] [12:14:14] <cremes> REP sockets know how to “route” their responses back over multiple hops
[Monday 13 September 2010] [12:14:29] <cremes> you need to do a little extra work when using an XREP socket to retain that functionality
[Monday 13 September 2010] [12:18:53] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [12:26:08] <ptrb> curiouser and curiouser
[Monday 13 September 2010] [12:26:48] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [12:39:20] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Monday 13 September 2010] [13:10:03] Quit bennymack has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [13:13:00] Quit ekidd has left this server (Quit: ekidd).
[Monday 13 September 2010] [13:19:56] Quit omarkj has left this server (Quit: omarkj).
[Monday 13 September 2010] [13:27:35] Join AlexB has joined this channel (
alex@brasetvik.com).
[Monday 13 September 2010] [14:12:09] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [14:13:16] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [14:45:08] Quit ReinH has left this server (Ping timeout: 245 seconds).
[Monday 13 September 2010] [15:59:18] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [16:08:23] Quit rphillips has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [16:37:48] <ModusPwnens> Hi guys, I have a question on req/rep topology
[Monday 13 September 2010] [16:38:17] <ModusPwnens> Previously, I have been doing benchmarking with the subscriber/publisher toplogy, but I wanted to see what results I would get with req/rep
[Monday 13 September 2010] [16:38:52] <ModusPwnens> and I was wondering if there is anything else I need to do besides the obvious change of the socket types and adding in addition send/recv function calls to avoid blocking the code
[Monday 13 September 2010] [16:39:26] <ModusPwnens> because i noticed after I did those things, rather than sending a message of X bytes, it sends X messages of 1 byte
[Monday 13 September 2010] [16:41:05] <ModusPwnens> Actually, I lied. It seems to just send a lot of zero byte messages
[Monday 13 September 2010] [16:41:49] <cremes> ModusPwnens: pastie some code, because it should “just work”
[Monday 13 September 2010] [16:44:17] Quit gavinstark has left this server (Remote host closed the connection).
[Monday 13 September 2010] [16:51:29] <ModusPwnens> Actually, i figured out what it was. However, should rep/req have better or worse performance than sub/pub?
[Monday 13 September 2010] [17:07:24] <cremes> ModusPwnens: same perf but round-trip latency is higher (no such notion as round-trip latency with pub/sub)
[Monday 13 September 2010] [17:27:18] Quit plq has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [18:07:29] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [18:33:14] Quit ekidd has left this server (Quit: ekidd).
[Monday 13 September 2010] [18:46:37] Quit psino has left this server (Quit: psino).
[Monday 13 September 2010] [19:10:47] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [19:57:38] Quit rgl has left this server (Ping timeout: 264 seconds).
[Monday 13 September 2010] [20:24:01] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [20:39:05] Quit rphillips has left this server (Ping timeout: 265 seconds).
[Monday 13 September 2010] [20:50:48] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [20:54:32] Quit jhawk28 has left this server (Ping timeout: 272 seconds).
[Monday 13 September 2010] [20:59:33] Quit
jhawk28_ has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [21:05:12] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [21:10:53] Quit plq has left this server (Ping timeout: 240 seconds).
[Monday 13 September 2010] [21:15:35] Quit andrewvc has left this server (Quit: andrewvc).
[Monday 13 September 2010] [21:42:44] Quit GeekGod has left this server (Quit: GeekGod).
[Monday 13 September 2010] [21:58:16] Quit grahamalot has left this server (Quit: grahamalot).
[Monday 13 September 2010] [22:00:53] Quit xla has left this server (Ping timeout: 276 seconds).
[Monday 13 September 2010] [22:17:53] Quit bgranger has left this server (Quit: bgranger).
[Monday 13 September 2010] [22:30:44] Quit bgranger has left this server (Read error: Connection reset by peer).
[Monday 13 September 2010] [22:45:29] Quit bgranger has left this server (Client Quit).
[Monday 13 September 2010] [23:39:45] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [00:52:17] Quit keffo has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [01:06:56] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [01:14:25] Quit psino has left this server (Quit: psino).
[Tuesday 14 September 2010] [01:55:59] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 14 September 2010] [01:56:36] Quit keffo has left this server (Read error: Connection reset by peer).
[Tuesday 14 September 2010] [01:58:14] Quit rgl has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [02:57:32] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [03:14:11] Quit jond has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [04:45:14] <pieterh> sustrik: ping?
[Tuesday 14 September 2010] [04:45:31] <sustrik> pieterh: pong
[Tuesday 14 September 2010] [04:45:42] <pieterh> is there any zero copy mechanism on reading messages?
[Tuesday 14 September 2010] [04:45:51] <pieterh> i.e. directly into application buffers
[Tuesday 14 September 2010] [04:46:45] <sustrik> no, you get the buffer from 0mq
[Tuesday 14 September 2010] [04:46:53] <sustrik> you can then store it as long as you wish
[Tuesday 14 September 2010] [04:47:12] <pieterh> ok, thx
[Tuesday 14 September 2010] [05:04:05] <ptrb> so you can
`zero-copy’ the buffer data beyond the lifetime of the zmq_msg_t?
[Tuesday 14 September 2010] [05:08:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [05:10:23] <sustrik> ptrb: no, but you can keep zmq_msg_t around as long as you wish
[Tuesday 14 September 2010] [05:10:34] <ptrb> roger.
[Tuesday 14 September 2010] [05:18:18] <pieterh> ptrb: I’ve added a problem solver to the guide
[Tuesday 14 September 2010] [05:20:46] <ptrb> cool
[Tuesday 14 September 2010] [05:21:00] <ptrb> the “Yes” path on “Are you using XREP sockets?” isn’t labeled :D
[Tuesday 14 September 2010] [05:21:08] <pieterh> thanks...!
[Tuesday 14 September 2010] [05:21:22] <ptrb> but that is definitely helpful
[Tuesday 14 September 2010] [05:21:40] <ptrb> i’m a little overwhelmed by the sheer number of different “guides” and “manuals” and whatnot on the docs page(s), though
[Tuesday 14 September 2010] [05:23:39] <pieterh> ptrb: yeah, standard process, lots of growth and then consolidation
[Tuesday 14 September 2010] [05:23:50] <pieterh> stick with www.zeromq.com for starters
[Tuesday 14 September 2010] [05:24:44] <pieterh> really there should only be two books, the guide and the reference manual
[Tuesday 14 September 2010] [05:24:55] <ptrb> sure
[Tuesday 14 September 2010] [05:25:16] <ptrb> there’s a solid case for a quick-start (~1h) tutorial as well, I think; maybe that’s like ch.1 of the guide
[Tuesday 14 September 2010] [05:25:39] <pieterh> yup
[Tuesday 14 September 2010] [05:25:55] <pieterh> plus lots of tutorials and articles from the rest of the community
[Tuesday 14 September 2010] [05:41:04] <keffo> pieterh, I have a concurrency issue that I cant really figure out!
[Tuesday 14 September 2010] [05:41:14] <pieterh> keffo: shoot
[Tuesday 14 September 2010] [05:41:31] <pieterh> have you checked the new problem solver?
[Tuesday 14 September 2010] [05:41:40] <keffo> lemme check
[Tuesday 14 September 2010] [05:42:57] <keffo> pieterh, Think client, double lru-queue(for outoing tasks, and for incoming results), then an additional ‘regular’ queue on each machine, which provides N worker processes..
[Tuesday 14 September 2010] [05:43:14] <keffo> pieterh, Theyre all sep. processes, no threading involved anywhere apart from zmq internals..
[Tuesday 14 September 2010] [05:43:22] <pieterh> ok
[Tuesday 14 September 2010] [05:43:53] <pieterh> what is the symptom you are seeing?
[Tuesday 14 September 2010] [05:44:29] <keffo> One worker, all is fine, I’ve done 100000 tasks back’n’forth with no problem, so the handling of data is sound.. Now if I do more than one worker processes, -sometimes- I get jumbled data, so it’s clearly a concurrency issue, but I cant really tell how or where
[Tuesday 14 September 2010] [05:45:12] <pieterh> jumbled data = one message has corrupted data or messages are not in order?
[Tuesday 14 September 2010] [05:45:34] <keffo> I guess both? =)
[Tuesday 14 September 2010] [05:45:43] <keffo> lemme trigger it and check the logs
[Tuesday 14 September 2010] [05:45:48] <pieterh> ok
[Tuesday 14 September 2010] [05:46:48] <keffo> there we go
[Tuesday 14 September 2010] [05:47:53] <keffo> so this scenario has multiple worker processes, but I send out batches or bursts of tasks of different granularity.. With 1 it always works, that is, send one task, wait for that task to complete.. The problem pops up when doing more than one, say 10
[Tuesday 14 September 2010] [05:48:00] <keffo> which is the case now
[Tuesday 14 September 2010] [05:48:31] <pieterh> keffo: you still need to explain what the “problem” actually is
[Tuesday 14 September 2010] [05:48:57] <pieterh> what is the difference between working and non-working in terms of what you see in the logs?
[Tuesday 14 September 2010] [05:49:43] <keffo> well, sometimes routing info is jumbled.. I expect uuid + nil + payload, but sometimes that uuid can get very weird
[Tuesday 14 September 2010] [05:50:01] <keffo> and likewise the payload itself can get messed up
[Tuesday 14 September 2010] [05:50:12] <pieterh> ok, so your message envelopes are wrong
[Tuesday 14 September 2010] [05:50:25] <keffo> They would be wrong with just one message too :)
[Tuesday 14 September 2010] [05:50:32] <keffo> or one worker proc.
[Tuesday 14 September 2010] [05:50:43] <pieterh> most probably an error in your code that creates/copies envelopers
[Tuesday 14 September 2010] [05:50:47] <pieterh>
*envelopes
[Tuesday 14 September 2010] [05:51:19] <pieterh> the LRU queue stuff is pretty delicate
[Tuesday 14 September 2010] [05:53:47] <keffo> I dont see the relevance to be honest, I can process 1M tasks properly if I wait for each message(or if I have only one worker processing them).. If there was an inherent problem in the envelope construction/usage, surely that would pop up there as well, but it has -never- happened in that scenario..
[Tuesday 14 September 2010] [05:54:02] Quit jsimmons has left this server (Ping timeout: 252 seconds).
[Tuesday 14 September 2010] [05:54:40] <pieterh> keffo: when problem solving you don’t care when it works
[Tuesday 14 September 2010] [05:54:52] <pieterh> if you can reproduce a problem, that’s the interesting thing
[Tuesday 14 September 2010] [05:55:06] <pieterh> things often work by accident...
[Tuesday 14 September 2010] [05:55:21] <pieterh> so your workers are REQ and you’re talking to them with an XREP?
[Tuesday 14 September 2010] [05:55:54] <keffo> yeah
[Tuesday 14 September 2010] [05:56:12] <pieterh> do you store the worker identity anywhere?
[Tuesday 14 September 2010] [05:56:23] <keffo> I can reproduce it, but with irregularities one has come to expect from multithreaded :)
[Tuesday 14 September 2010] [05:56:31] <keffo> yeah, all connections are known
[Tuesday 14 September 2010] [05:57:24] <pieterh> so the obvious thing to look at would be how you reconstruct the envelopes and where the worker addresses come from
[Tuesday 14 September 2010] [05:57:32] <pieterh> and that you’re not using the wrong data
[Tuesday 14 September 2010] [05:57:44] Quit omarkj has left this server (Read error: Connection reset by peer).
[Tuesday 14 September 2010] [05:58:34] <pieterh> you presumably have logic that is vulnerable to multiple workers sending back stuff at the same time
[Tuesday 14 September 2010] [05:59:04] <pieterh> maybe you’re not using SNDMORE properly in the worker when sending
[Tuesday 14 September 2010] [05:59:22] <pieterh> so messages arrive mixed together rather than atomically
[Tuesday 14 September 2010] [05:59:31] * pieterh has no idea if that is even possible...
[Tuesday 14 September 2010] [05:59:53] <keffo> right, I dont see that being possible either, as it properly routes across 3-4 queues?
[Tuesday 14 September 2010] [06:00:08] <pieterh> you have assertions on everything?
[Tuesday 14 September 2010] [06:00:20] <keffo> think so, yeah
[Tuesday 14 September 2010] [06:00:34] <pieterh> so every single frame is checked? e.g. RCVMORE where expected, empty frame where expected, etc.
[Tuesday 14 September 2010] [06:00:50] <keffo> yes
[Tuesday 14 September 2010] [06:01:05] <pieterh> the only think I can think of right now is that you can send what looks like a valid envelope but it won’t survive concurrency
[Tuesday 14 September 2010] [06:02:21] <keffo> What would cause multiple incoming multipart-messages to break symmetry with more than one sender, in a singlethreaded process?
[Tuesday 14 September 2010] [06:02:34] <keffo> that’s what I dont get :)
[Tuesday 14 September 2010] [06:02:47] <pieterh> well, if they are not actually multipart messages
[Tuesday 14 September 2010] [06:02:56] <pieterh> lets say you send the envelope (address, null, data)
[Tuesday 14 September 2010] [06:03:06] <pieterh> but you by mistake use normal sends on each part
[Tuesday 14 September 2010] [06:03:19] <keffo> yup
[Tuesday 14 September 2010] [06:03:21] <pieterh> it’ll be 3 messages, not 1 3-part message
[Tuesday 14 September 2010] [06:03:31] <pieterh> if you read that, it might look valid and work for routing
[Tuesday 14 September 2010] [06:03:40] <pieterh> but as soon as you have concurrency it’ll stop working
[Tuesday 14 September 2010] [06:03:56] <pieterh> like i say, i’ve never tried that, but will make a test now, after coffee
[Tuesday 14 September 2010] [06:04:12] <pieterh> i’d expect 0MQ to reject such messages...
[Tuesday 14 September 2010] [06:04:29] <keffo> hmm.. I’ll go over the code again! :)
[Tuesday 14 September 2010] [06:04:32] <pieterh> but from REQ to XREP it can’t IMO
[Tuesday 14 September 2010] [06:06:40] <keffo> cant reject?
[Tuesday 14 September 2010] [06:07:04] <sustrik> keffo: can you check whether the garbled identity you see is something that may actually your application message or whether it’s just real junk
[Tuesday 14 September 2010] [06:07:05] <sustrik> ?
[Tuesday 14 September 2010] [06:09:20] <keffo> It’s not real junk.. Sometimes for example I get route uuid that appears to be a mix of multiple uuid’s
[Tuesday 14 September 2010] [06:09:41] <keffo> clearly something breaks symmetry somehow
[Tuesday 14 September 2010] [06:09:50] <keffo> I’ll lok into the route logic again though
[Tuesday 14 September 2010] [06:11:56] <sustrik> “mix of multiple uuid’s” = junk IMO
[Tuesday 14 September 2010] [06:12:08] <sustrik> ok, a sanity check:
[Tuesday 14 September 2010] [06:12:20] <sustrik> do you use each socket exclusively from a single thread?
[Tuesday 14 September 2010] [06:12:44] <keffo> yeah
[Tuesday 14 September 2010] [06:12:54] <sustrik> what language are you using?
[Tuesday 14 September 2010] [06:13:02] <keffo> c++ and lua
[Tuesday 14 September 2010] [06:13:10] <keffo> but homebrew bindings
[Tuesday 14 September 2010] [06:13:42] <sustrik> any chance to test a pure c++ setup?
[Tuesday 14 September 2010] [06:13:55] <keffo> hehe. noo :)
[Tuesday 14 September 2010] [06:14:35] <sustrik> hm, lua binding is a project i know nothing about
[Tuesday 14 September 2010] [06:15:02] <sustrik> the bug can be there, as no such problem was yet reported for any other language
[Tuesday 14 September 2010] [06:15:30] <sustrik> ok, any chance of using anything other than lua?
[Tuesday 14 September 2010] [06:19:11] <keffo> Would take forever to rewrite
[Tuesday 14 September 2010] [06:19:20] <sustrik> ok
[Tuesday 14 September 2010] [06:19:32] <sustrik> how does the setup look like?
[Tuesday 14 September 2010] [06:19:40] <sustrik> which part is c++ and which is lua?
[Tuesday 14 September 2010] [06:20:03] <keffo> all zmq stuff is c, rest is lua.. Ie, connect/bind/send/recv(and their more-counterparts)
[Tuesday 14 September 2010] [06:20:51] <sustrik> you are mixing C code and lua code to use the same 0MQ socket?
[Tuesday 14 September 2010] [06:21:01] * sustrik doesn’t follow
[Tuesday 14 September 2010] [06:21:22] <keffo> Um no, I created bindings for sockets, so it’s the same thing
[Tuesday 14 September 2010] [06:21:50] <sustrik> hm, are you saying that 0MQ is written in C while your apps are in lua?
[Tuesday 14 September 2010] [06:22:29] <keffo> I’ve extended an application with lua yeah..
[Tuesday 14 September 2010] [06:22:34] <keffo> lua or not is besides the point really
[Tuesday 14 September 2010] [06:22:47] <keffo> All zmq related code is in C++
[Tuesday 14 September 2010] [06:23:24] <sustrik> aha, so you are _not_ using the lua binding
[Tuesday 14 September 2010] [06:23:26] <sustrik> right?
[Tuesday 14 September 2010] [06:23:44] <keffo> I am, but not the ones from the site :)
[Tuesday 14 September 2010] [06:23:50] <sustrik> your own
[Tuesday 14 September 2010] [06:23:55] <keffo> yup
[Tuesday 14 September 2010] [06:24:14] <keffo> just an abstraction layer that’s easier to work with on the lua side
[Tuesday 14 September 2010] [06:24:20] <sustrik> ack
[Tuesday 14 September 2010] [06:24:29] <sustrik> what’s the lua’s threading model?
[Tuesday 14 September 2010] [06:24:40] <keffo> inherently single threaded, but it has coroutines
[Tuesday 14 September 2010] [06:25:38] <sustrik> ok, so there’s only one OS thread for each lua process, right?
[Tuesday 14 September 2010] [06:25:42] <keffo> yes
[Tuesday 14 September 2010] [06:26:22] <sustrik> no such think as garbage collection thread or similar?
[Tuesday 14 September 2010] [06:26:25] <sustrik> thing*
[Tuesday 14 September 2010] [06:26:26] <keffo> client, loadbalancer, computenode and workers are all single processes (a computenode is simply a c++ queue dev. ‘owning’ the worker processes)
[Tuesday 14 September 2010] [06:26:52] <keffo> No, not in another thread no.. Lua does incremental gc but in the same thread
[Tuesday 14 September 2010] [06:27:11] <sustrik> ok
[Tuesday 14 September 2010] [06:27:37] <sustrik> then it looks like the problem has to be in 0MQ itself
[Tuesday 14 September 2010] [06:27:46] <sustrik> it should be reproducible in C or C++
[Tuesday 14 September 2010] [06:28:15] <sustrik> can you write a simpe C/C++ test that simulates what you are doing in your app?
[Tuesday 14 September 2010] [06:28:35] <sustrik> messaging-wise i mean, not the business logic
[Tuesday 14 September 2010] [06:28:47] <keffo> hmm.. that would take a while I think..
[Tuesday 14 September 2010] [06:29:07] <keffo> I do think I’m to blame, not zmq, I just need a better way to debug it probably :)
[Tuesday 14 September 2010] [06:30:01] <keffo> It behaves as threaded concurrency issue, but being entirely single process based..
[Tuesday 14 September 2010] [06:30:37] <sustrik> there’s an I/O thread inside 0MQ so it can be a race
[Tuesday 14 September 2010] [06:31:00] <sustrik> however, i can’t help unless i am able to reproduce it
[Tuesday 14 September 2010] [06:31:20] <keffo> Well I’d be half way safe if I could reproduce it too, but it’s quite rare :)
[Tuesday 14 September 2010] [06:31:37] <sustrik> of course, it’s still possible that you are overwriting memory in your app or something like that
[Tuesday 14 September 2010] [06:31:54] <sustrik> have you tried to valgring your binding?
[Tuesday 14 September 2010] [06:31:58] <sustrik> valgrind
[Tuesday 14 September 2010] [06:33:17] <keffo> no valgrind in win as far as I know.. Boundschecker perhaps, but I doubt that’s the problem too
[Tuesday 14 September 2010] [06:33:41] <keffo> Again, in a single-worker scenario, everything is flawless regardless of amount and load etc
[Tuesday 14 September 2010] [06:34:00] <keffo> (and none of the processes every leak anything, stays solid around 2mb mem)
[Tuesday 14 September 2010] [06:34:39] <sustrik> then it looks like a problem in the central dispatcher node
[Tuesday 14 September 2010] [06:34:50] <keffo> yeah
[Tuesday 14 September 2010] [06:34:57] <keffo> that’s the most complicated bit too
[Tuesday 14 September 2010] [06:35:09] <keffo> double lru queue device basically
[Tuesday 14 September 2010] [06:36:01] <sustrik> well, unless you are able to provide something to reproduce the problem, you are on your own here
[Tuesday 14 September 2010] [06:36:08] <keffo> I know :)
[Tuesday 14 September 2010] [06:37:02] <keffo> more logging I guess. It would be nice to have all participants publish all the data they process, and collect that in a sub app, and then be able to “play back” everything
[Tuesday 14 September 2010] [06:37:30] <ptrb> that could easily become a project as complex as zmq itself
[Tuesday 14 September 2010] [06:37:43] <ptrb> entire companies are built on the concept of capturing/replaying network traffic
[Tuesday 14 September 2010] [06:37:52] <pieterh> keffo: if you can send me your code I’ll see if I can spot anything
[Tuesday 14 September 2010] [06:38:03] <pieterh> i need the parts that read and write worker envelopes, to start with
[Tuesday 14 September 2010] [06:38:12] <keffo> sure hold on
[Tuesday 14 September 2010] [06:38:17] <keffo> wheres the pastebin?
[Tuesday 14 September 2010] [06:38:29] <pieterh> gist.github.com works well
[Tuesday 14 September 2010] [06:39:04] <keffo> But again, if the envelope code is flawed, it would be equally flawed for one worker?
[Tuesday 14 September 2010] [06:40:27] <pieterh> keffo: i explained how it could be flawed but still work for one worker
[Tuesday 14 September 2010] [06:42:22] <pieterh> keffo: i need the actual detail of sendpath, I think
[Tuesday 14 September 2010] [06:42:27] <pieterh>
*sendmore, sorry
[Tuesday 14 September 2010] [06:42:45] <keffo> That is in C, hold on
[Tuesday 14 September 2010] [06:44:52] <keffo> send() is obviously quite similar except for the flag :)
[Tuesday 14 September 2010] [06:45:56] <pieterh> keffo: it looks fine
[Tuesday 14 September 2010] [06:46:15] <pieterh> you’re sure that pairs() gives you stuff in the right order?
[Tuesday 14 September 2010] [06:46:38] <pieterh> so you have only one choice left, sadly
[Tuesday 14 September 2010] [06:46:55] <keffo> rewrite, and hope it goes away? =)
[Tuesday 14 September 2010] [06:47:05] <keffo> naa, there’s some stupid silly thing somewhere
[Tuesday 14 September 2010] [06:47:13] <pieterh> well, everything goes through these functions
[Tuesday 14 September 2010] [06:47:20] <pieterh> so put in printf statements or whatever
[Tuesday 14 September 2010] [06:47:27] <pieterh> and see when it actually goes wrong
[Tuesday 14 September 2010] [06:47:46] <keffo> Yeah, need to find a pattern..
[Tuesday 14 September 2010] [06:47:50] <pieterh> pieter’s rules #5 and #23 of debugging
[Tuesday 14 September 2010] [06:48:03] <pieterh> (a) it’s never actually a race condition
[Tuesday 14 September 2010] [06:48:14] <pieterh> (b) you can find any bug using printfs
[Tuesday 14 September 2010] [06:48:23] <keffo> except the ones in printf!
[Tuesday 14 September 2010] [06:48:37] <pieterh> that’s why C offers puts as well!
[Tuesday 14 September 2010] [06:49:39] <keffo> One thing I could do is rewrite the loadbalancer stuff in pure C
[Tuesday 14 September 2010] [06:50:11] <pieterh> keffo: there’s no point rewriting unless you have found the cause of the problem
[Tuesday 14 September 2010] [06:50:31] <ptrb> why are your rule #5 and #23 labelled (a) and (b)??
[Tuesday 14 September 2010] [06:50:31] <keffo> indeed
[Tuesday 14 September 2010] [06:51:04] <pieterh> ptrb: lol
[Tuesday 14 September 2010] [06:51:04] <keffo> pieterh, That code should be rewritten though, for performance, but I agree :)
[Tuesday 14 September 2010] [07:33:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [07:37:02] <BooTheHamster> What mean error “operation cannot be accomplished in current state” in zmq_send() call?
[Tuesday 14 September 2010] [07:38:51] <ptrb> hey i got that yesterday! :D
[Tuesday 14 September 2010] [07:39:00] <ptrb> basically means you’re violating some constraint of the socket
[Tuesday 14 September 2010] [07:39:08] <ptrb> is it ZMQ_REQ or _REP?
[Tuesday 14 September 2010] [07:39:15] <BooTheHamster> ZMQ_REQ
[Tuesday 14 September 2010] [07:39:37] <ptrb> you must call zmq_send, then zmq_recv, before you can call zmq_send again
[Tuesday 14 September 2010] [07:39:39] <BooTheHamster> I’t a client app but run withiut server
[Tuesday 14 September 2010] [07:39:39] <ptrb> are you doing that?
[Tuesday 14 September 2010] [07:39:46] <BooTheHamster> yes
[Tuesday 14 September 2010] [07:39:55] <BooTheHamster> first I call zmq_send
[Tuesday 14 September 2010] [07:40:05] <BooTheHamster> and then zmq_recv
[Tuesday 14 September 2010] [07:40:11] <ptrb> on the same socket
[Tuesday 14 September 2010] [07:40:19] <BooTheHamster> m_MsgSock->recv(&msg_rcv, ZMQ_NOBLOCK)
[Tuesday 14 September 2010] [07:40:21] <BooTheHamster> yes
[Tuesday 14 September 2010] [07:40:33] <ptrb> noblock means you may not actually have recv’d anything
[Tuesday 14 September 2010] [07:40:46] <ptrb> you have to actually recv something before you can send again (I believe–maybe someone can confirm)
[Tuesday 14 September 2010] [07:41:21] <BooTheHamster> but i want non blocking recv
[Tuesday 14 September 2010] [07:41:36] <ptrb> that’s fine, you just can’t send again until your non-blocking recv actually gets something
[Tuesday 14 September 2010] [07:41:57] <ptrb> if you want to do multiple sends without getting data back, use XREQ
[Tuesday 14 September 2010] [07:43:41] <BooTheHamster> Thank you ptrb, XREQ helps.
[Tuesday 14 September 2010] [07:44:29] <BooTheHamster> A little one question ... on server side i must use XREP or REP socket?
[Tuesday 14 September 2010] [07:45:36] <ptrb> just depends on what behavior you can tolerate there... both will work, but REP requires you to recv, send, recv, send actual data
[Tuesday 14 September 2010] [07:46:08] <ptrb>
`man zmq_socket’ for more details
[Tuesday 14 September 2010] [07:50:49] <BooTheHamster> thanks
[Tuesday 14 September 2010] [07:52:34] Part BooTheHamster has left this channel.
[Tuesday 14 September 2010] [07:52:40] <ptrb> i helped!! :D
[Tuesday 14 September 2010] [07:52:44] <ptrb> maybe
[Tuesday 14 September 2010] [07:55:42] Quit psino has left this server (Quit: psino).
[Tuesday 14 September 2010] [08:04:35] Part icy has left this channel.
[Tuesday 14 September 2010] [08:25:25] <cremes> ptrb: you’re an expert now! ;)
[Tuesday 14 September 2010] [08:46:18] Quit jsimmons has left this server (Quit: Leaving).
[Tuesday 14 September 2010] [08:48:39] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Tuesday 14 September 2010] [08:51:23] Quit xla has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [09:02:00] <pieterh> cremes: I moved your xrep-xrep recipe page into the tutorials section
[Tuesday 14 September 2010] [09:07:28] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 14 September 2010] [09:27:17] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [09:47:35] <Tasser> why use rbzmq <=> ffi-rzmq?
[Tuesday 14 September 2010] [09:54:46] <cremes> Tasser: they have slightly different APIs; the ffi-rzmq more closely tracks the C api whereas rbzmq uses strings for everything
[Tuesday 14 September 2010] [09:55:02] <cremes> also, ffi-rzmq is more compatible with ruby runtimes (jruby, rubinius, mri)
[Tuesday 14 September 2010] [09:55:08] <Tasser> strings sounds like overuse of core classes
[Tuesday 14 September 2010] [09:55:37] <cremes> well, most people will probably convert their data to string format for send/recv under ruby because it’s convenient
[Tuesday 14 September 2010] [09:55:54] <Tasser> that’s what JSON is for :-)
[Tuesday 14 September 2010] [09:56:10] <cremes> but since ffi-rzmq exposes a zmq_msg_t-like structure you can pass around C structs without doing a lot of extra copying
[Tuesday 14 September 2010] [09:56:15] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [09:56:15] <Tasser> but I suspect you can’t hardwire zmq against json
[Tuesday 14 September 2010] [09:56:18] <cremes> that might be more efficient
[Tuesday 14 September 2010] [09:56:40] <cremes> what do you mean by hardware? as far as i know, 0mq (and zmq) don’t care what you pass on the wire
[Tuesday 14 September 2010] [09:56:47] <cremes> s/hardware/hardwire/
[Tuesday 14 September 2010] [09:57:07] <Tasser> well, for filtering in subscribe
[Tuesday 14 September 2010] [09:57:28] <cremes> ah, subscribe filters at the byte-level
[Tuesday 14 September 2010] [09:57:55] <cremes> no need to encode the subscription topic unless you want to decode it on each subscriber too
[Tuesday 14 September 2010] [09:59:20] <Tasser> hmm
[Tuesday 14 September 2010] [10:00:50] <cremes> hmm sometimes means “i don’t know what you are talking about” :)
[Tuesday 14 September 2010] [10:10:21] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [10:51:56] Quit bgranger has left this server (Client Quit).
[Tuesday 14 September 2010] [10:58:00] <Tasser> cremes, well, so you could filter based on JSON, but that would make it bloat
[Tuesday 14 September 2010] [10:58:28] Join jond has joined this channel (
~jon@87.74.16.24).
[Tuesday 14 September 2010] [10:59:41] <cremes> Tasser: my point is that 0mq matches topics by comparing bytes
[Tuesday 14 September 2010] [10:59:48] <cremes> so you want it to fail as soon as possible
[Tuesday 14 September 2010] [11:00:05] <cremes> using a json-encoded topic means that more bytes would need to be compared to find mismatches
[Tuesday 14 September 2010] [11:00:13] <cremes> so subscription filtering would be slower
[Tuesday 14 September 2010] [11:00:23] <cremes> for example...
[Tuesday 14 September 2010] [11:00:51] <cremes> sock.setsockopt ZMQ::SUBSCRIBE, ‘topic.string’
[Tuesday 14 September 2010] [11:00:53] <cremes> versus
[Tuesday 14 September 2010] [11:01:25] <cremes> sock.setsockopt(ZMQ::SUBSCRIBE, JSON.encode({‘topic’ => ‘topic.string’})
[Tuesday 14 September 2010] [11:01:36] <cremes> honestly, i’m not even sure if that second example would ever work
[Tuesday 14 September 2010] [11:01:53] <cremes> since the json encoder could put things in different order while still producing valid json
[Tuesday 14 September 2010] [11:02:10] <cremes> just use a string for the topic and put your encoded data into a second message part
[Tuesday 14 September 2010] [11:02:48] <Tasser> yeah, probably the sane idea of using it.
[Tuesday 14 September 2010] [11:50:19] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Tuesday 14 September 2010] [11:53:31] Quit zoobab has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [12:05:40] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [12:11:28] Quit jdknezek has left this server (Quit: Page closed).
[Tuesday 14 September 2010] [12:36:34] Quit
keffo_ has left this server (Ping timeout: 265 seconds).
[Tuesday 14 September 2010] [13:00:39] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Tuesday 14 September 2010] [13:01:00] <kleppari> hello
[Tuesday 14 September 2010] [13:02:38] <ModusPwnens> Im not sure, but if you do register, let me know if you are successful. I was trying to the other day and it wouldn’t work =/
[Tuesday 14 September 2010] [13:03:01] <pieterh> kleppari: isn’t is showing? rats... :-( thanks for pointing it out
[Tuesday 14 September 2010] [13:03:08] <guido_g> f*ck, this is wikidot crap
[Tuesday 14 September 2010] [13:03:14] <guido_g> sorry
[Tuesday 14 September 2010] [13:03:32] <pieterh> gudio_g: default permissions on cloned site are private, I forgot
[Tuesday 14 September 2010] [13:03:54] <guido_g> and you need to allow cross-domain cookies
[Tuesday 14 September 2010] [13:03:57] <pieterh> happily it’s a 10-second fix
[Tuesday 14 September 2010] [13:04:09] <guido_g> wikidot sucks in this regard
[Tuesday 14 September 2010] [13:04:10] <pieterh> guido_g: you need to not disallow the dratted things
[Tuesday 14 September 2010] [13:04:21] <pieterh> it’s for good security reasons, unfortunately
[Tuesday 14 September 2010] [13:04:32] <kleppari> registered with wikidots, worked after that. I think it’s a major fail to require it though :)
[Tuesday 14 September 2010] [13:04:43] <pieterh> kleppari: not required, it was my fault
[Tuesday 14 September 2010] [13:04:44] <kleppari> pieterh: np, thanks for the guide :)
[Tuesday 14 September 2010] [13:04:49] <guido_g> pieterh: cross-donmain cookies are for security?
[Tuesday 14 September 2010] [13:04:50] <pieterh> it should work for anonymous users now
[Tuesday 14 September 2010] [13:05:08] <pieterh> guido_g: wikidot splits off insecure stuff (file uploads) to a separate domain
[Tuesday 14 September 2010] [13:05:08] <guido_g> nack
[Tuesday 14 September 2010] [13:05:30] <guido_g> pieterh: it doesen’t work
[Tuesday 14 September 2010] [13:05:30] <pieterh> feel free to debate with the wikidot devs, they are pretty sure of their skills in this respect
[Tuesday 14 September 2010] [13:05:46] <pieterh> what doesn’t work?
[Tuesday 14 September 2010] [13:05:48] <kleppari> pieterh: works now, thanks.
[Tuesday 14 September 2010] [13:05:52] <pieterh> ?
[Tuesday 14 September 2010] [13:06:03] <kleppari> i.e., anonymous users on zguide.zeromq.org
[Tuesday 14 September 2010] [13:06:11] <kleppari> oh, no, sorry
[Tuesday 14 September 2010] [13:06:14] <guido_g> w/ the damn cookies
[Tuesday 14 September 2010] [13:06:51] <pieterh> guido_g: anything different than zeromq.org?
[Tuesday 14 September 2010] [13:07:03] <pieterh> there’s nothing special about this site afaics
[Tuesday 14 September 2010] [13:07:06] <Samy> sustrik, ping?
[Tuesday 14 September 2010] [13:07:21] <guido_g> i allowed the cookies for zeromq.org and the wikidot server
[Tuesday 14 September 2010] [13:07:49] <guido_g> but i’m not going to do that for every new site
[Tuesday 14 September 2010] [13:07:55] <pieterh> ah, well, allow them for
*.zeromq.org and
*.zeromq.com :-)
[Tuesday 14 September 2010] [13:08:08] <pieterh> why do you disallow them?
[Tuesday 14 September 2010] [13:08:18] <guido_g> i do not allow cookies generally
[Tuesday 14 September 2010] [13:08:26] <pieterh> well...
[Tuesday 14 September 2010] [13:08:48] <pieterh> kleppari: thanks, I’ll try it...
[Tuesday 14 September 2010] [13:08:51] * pieterh recalls people saying “I don’t use JavaScript”
[Tuesday 14 September 2010] [13:09:14] <pieterh> unfortunately interactive websites kind of need stuff like that
[Tuesday 14 September 2010] [13:09:43] <guido_g> viewing a page is not interactive
[Tuesday 14 September 2010] [13:10:08] <kleppari> sorry, screwed up file ownership. the screenshot of unauthenticated zguide.zeromq.org should be ok now.
[Tuesday 14 September 2010] [13:10:41] <pieterh> kleppari, hang on, 30 seconds... I’ll fix it, thanks for the screenshot
[Tuesday 14 September 2010] [13:11:10] <pieterh> can you try to reload the site please?
[Tuesday 14 September 2010] [13:11:13] <pieterh> ctrl-R
[Tuesday 14 September 2010] [13:11:23] <kleppari> like a glove
[Tuesday 14 September 2010] [13:11:25] <kleppari> thanks :)
[Tuesday 14 September 2010] [13:11:39] <pieterh> i didn’t change anything, it was already working... :-)
[Tuesday 14 September 2010] [13:11:50] <pieterh> since 19:03 CET
[Tuesday 14 September 2010] [13:12:10] <pieterh> ok, kleppari I appreciate the heads up, kind of stupid to have a Guide no-one can read... :-)
[Tuesday 14 September 2010] [13:12:26] <kleppari> screenshot snapped at 17:06 GMT. Weird.
[Tuesday 14 September 2010] [13:12:32] <kleppari> you’re welcome
[Tuesday 14 September 2010] [13:12:56] <pieterh> guido_g, yeah, because Sustrik pointed out that parts of it were bogus
[Tuesday 14 September 2010] [13:13:00] <pieterh> i’m rewriting it
[Tuesday 14 September 2010] [13:13:05] <pieterh> as soon as i finish my pasta and wine
[Tuesday 14 September 2010] [13:13:09] <guido_g> and why this intermediate page when i click in “The Guide” from zeromq.org?
[Tuesday 14 September 2010] [13:13:37] <pieterh> you’d rather leap to chapter 1 right away?
[Tuesday 14 September 2010] [13:13:48] <guido_g> yes
[Tuesday 14 September 2010] [13:13:55] <pieterh> i mean when you get a book, you see the cover first...
[Tuesday 14 September 2010] [13:14:07] <pieterh> but ok, can be done...
[Tuesday 14 September 2010] [13:14:08] <guido_g> the link is named “The Guide”
[Tuesday 14 September 2010] [13:14:24] <pieterh> Yeah, not The Guide Chapter 1 :-)
[Tuesday 14 September 2010] [13:14:36] <guido_g> not “promotional intermediate page w/o content”
[Tuesday 14 September 2010] [13:14:54] <pieterh> hey, don’t mock the promotional pages...
[Tuesday 14 September 2010] [13:15:22] <guido_g> only if you reduce the size of the MQ logo drastically
[Tuesday 14 September 2010] [13:15:25] <pieterh> delayed gratification...
[Tuesday 14 September 2010] [13:15:40] <pieterh> OMG don’t touch the logo, it’s sacred...
[Tuesday 14 September 2010] [13:15:44] <guido_g> btw, too much red makes aggressive :)
[Tuesday 14 September 2010] [13:15:53] <pieterh> ok, the link from the sidebad should now go straight to the first chapter
[Tuesday 14 September 2010] [13:16:20] <guido_g> nope
[Tuesday 14 September 2010] [13:16:33] * pieterh is on a lousy 2G modem that takes ages
[Tuesday 14 September 2010] [13:16:55] <guido_g> ok, i’ll wait
[Tuesday 14 September 2010] [13:17:20] <pieterh> guido_g, when you say “nope” you mean as in “oops, I didn’t reload it and HTTP doesn’t yet have a autoupdate function”...?
[Tuesday 14 September 2010] [13:17:57] <guido_g> pieterh: no, it means “i pressed relaod like maniac but the thing didn’t work”
[Tuesday 14 September 2010] [13:18:09] <pieterh> works for me...
[Tuesday 14 September 2010] [13:18:22] <pieterh> go back to www.zeromq.org
[Tuesday 14 September 2010] [13:18:46] Quit speedy1 has left this server (Quit: Leaving).
[Tuesday 14 September 2010] [13:18:47] <pieterh> are there more links? I fixed the one in the sidebar
[Tuesday 14 September 2010] [13:18:51] <pieterh> what one are you clicking on?
[Tuesday 14 September 2010] [13:18:54] <guido_g> i am on zeromq.org and open the link in another tab
[Tuesday 14 September 2010] [13:19:04] <guido_g> after reloading the page
[Tuesday 14 September 2010] [13:19:49] <pieterh> shrug...it’s correct... you are reloading zeromq.org?
[Tuesday 14 September 2010] [13:19:57] <guido_g> yes
[Tuesday 14 September 2010] [13:20:16] <guido_g> hmmm... cdn thing maybe
[Tuesday 14 September 2010] [13:20:25] <pieterh> did you sacrifice a chicken to the gnomes of the Internet?
[Tuesday 14 September 2010] [13:20:53] <guido_g> ahhh... i constantly miss this one (eating the chicken myself)
[Tuesday 14 September 2010] [13:21:07] <pieterh> might be some caching somewhere your side of things
[Tuesday 14 September 2010] [13:21:19] <pieterh> it’s correct, anyhow, just tested again
[Tuesday 14 September 2010] [13:21:31] <guido_g> ahhh... cdn got the update
[Tuesday 14 September 2010] [13:21:35] <kleppari> you might also be clicking different links?
[Tuesday 14 September 2010] [13:21:40] <guido_g> no
[Tuesday 14 September 2010] [13:22:09] <kleppari> the sidebar link points to chapter one, the other guide links point to the ‘promotional’ page
[Tuesday 14 September 2010] [13:22:27] * pieterh looks for the other guide links
[Tuesday 14 September 2010] [13:22:29] <guido_g> so remember, it might take approx. 5 minutes til changes become vsisible
[Tuesday 14 September 2010] [13:22:41] <kleppari> two of them, one under most popular, other in recent site changes
[Tuesday 14 September 2010] [13:22:51] <pieterh> ah, kleppari, right
[Tuesday 14 September 2010] [13:23:19] <kleppari> but.. After a little while I like the promotional page
[Tuesday 14 September 2010] [13:23:28] <pieterh> well, indeed, a book deserves a cover
[Tuesday 14 September 2010] [13:23:40] <pieterh> bright red
[Tuesday 14 September 2010] [13:24:17] <pieterh> anyhow, this is always a work in progress, i like to see how people respond over time before making more changes
[Tuesday 14 September 2010] [13:24:33] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 14 September 2010] [13:24:48] <kleppari> I didn’t get it at first though, started at zeromq.com -> hit read the manual -> clicked ‘MQ - The guide’ -> got to zguide.zeromq.org
[Tuesday 14 September 2010] [13:24:51] <guido_g> pieterh: you’re on a laptop?
[Tuesday 14 September 2010] [13:25:01] <kleppari> didn’t catch the differences between zguide.zeromq.org and zeromq.com right away
[Tuesday 14 September 2010] [13:25:13] <pieterh> guido_g, on a laptop with a 2G phone modem, eating pasta in a hotel
[Tuesday 14 September 2010] [13:25:26] <guido_g> it’S more about the display size
[Tuesday 14 September 2010] [13:25:32] <kleppari> sounds like a crap hotel :P
[Tuesday 14 September 2010] [13:25:53] <pieterh> kleppari, it’s a pretty amazing hotel actually, an old Soviet relic
[Tuesday 14 September 2010] [13:26:04] <pieterh> guido_g, yes, laptop display
[Tuesday 14 September 2010] [13:26:38] <guido_g> pieterh: because on my 13” display the logo takes up apporx. 1/3 of the vertical space
[Tuesday 14 September 2010] [13:26:51] <guido_g> which is way to much, imho
[Tuesday 14 September 2010] [13:27:08] <guido_g> worse if i zoom in for better readability
[Tuesday 14 September 2010] [13:27:18] <pieterh> guido_g, yes, on mine too but it doesn’t matter on the cover page, and hardly matters on the content pages since you need to scroll anyhow
[Tuesday 14 September 2010] [13:27:33] <kleppari> looks good on 1280x1024
[Tuesday 14 September 2010] [13:27:42] <pieterh> we tried using a smaller logo some time ago, people felt it wasn’t nice
[Tuesday 14 September 2010] [13:28:01] <kleppari> pieterh: and no rj45 jack? URL? :)
[Tuesday 14 September 2010] [13:28:20] <guido_g> ok, i give up
[Tuesday 14 September 2010] [13:28:27] <pieterh> kleppari, URL? no rj45, no, wifi but it’s too slow
[Tuesday 14 September 2010] [13:28:48] <kleppari> url of the hotel website? or is it too much of a relic?
[Tuesday 14 September 2010] [13:28:49] <pieterh> guido_g, do start a discussion on look and feel, but this isn’t the place IMO
[Tuesday 14 September 2010] [13:29:23] <pieterh> guido_g, the original look was designed by a proper artist, and anything that replaces it would have to be done carefully
[Tuesday 14 September 2010] [13:29:32] <pieterh> kleppari, it’s the Hotel Kyjev in Bratislava
[Tuesday 14 September 2010] [13:30:38] <guido_g> pieterh: i didn’t complain about the logo, only it’s size, but nevermind i’ll find a solution
[Tuesday 14 September 2010] [13:30:57] <pieterh> guido_g, I do agree the logo is HUGE but my previous attempts to downsize it failed
[Tuesday 14 September 2010] [13:31:23] <guido_g> ok
[Tuesday 14 September 2010] [13:32:11] <pieterh> guido_g, what might work is a lighter theme for sites other than the Start Here site
[Tuesday 14 September 2010] [13:32:23] <pieterh> now that we have more sites, we can fine-tune them
[Tuesday 14 September 2010] [13:38:04] <guido_g> except for the “promo” pages the focus should be on readability
[Tuesday 14 September 2010] [13:42:05] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [13:53:08] Quit
pieterh_ has left this server (Read error: No route to host).
[Tuesday 14 September 2010] [14:52:29] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 14 September 2010] [14:54:31] Quit xla has left this server (Ping timeout: 255 seconds).
[Tuesday 14 September 2010] [15:29:26] Quit
pieterh__ has left this server (Ping timeout: 264 seconds).
[Tuesday 14 September 2010] [16:18:53] Quit benoitc has left this server (Ping timeout: 245 seconds).
*** on Tue Sep 14 16:37:22 2010
[Tuesday 14 September 2010] [16:37:22] Topic The channel topic is “Welcome!”.
[Tuesday 14 September 2010] [16:37:28] Mode Channel modes: no messages from outside, topic protection
[Tuesday 14 September 2010] [16:37:28] Created This channel was created on 2010-02-05 10:44.
[Tuesday 14 September 2010] [16:44:44] Part bobs has left this channel.
[Tuesday 14 September 2010] [17:10:43] Quit rphillips has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [18:06:56] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [18:43:28] Quit ekidd has left this server (Quit: ekidd).
[Tuesday 14 September 2010] [18:57:59] Quit plq has left this server (Ping timeout: 276 seconds).
[Tuesday 14 September 2010] [19:40:23] Quit bgranger has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [19:44:27] Quit
bgranger_ has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [20:42:43] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [21:50:01] <kenkeiter> Okay, Rubyists with zmq experience – question!
[Tuesday 14 September 2010] [21:55:01] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [21:55:09] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [22:20:34] Quit AlexB has left this server (Remote host closed the connection).
[Tuesday 14 September 2010] [22:20:38] Join AlexB has joined this channel (
alex@brasetvik.com).
[Tuesday 14 September 2010] [22:23:00] <
Xin_> test
[Tuesday 14 September 2010] [22:25:21] <
Xin_> Does it make sense to add UDP support to 0mq?
[Tuesday 14 September 2010] [22:27:20] Quit
Xin_ has left this server (Quit: Page closed).
[Tuesday 14 September 2010] [22:47:48] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Tuesday 14 September 2010] [22:56:32] Quit kenkeiter has left this server (Quit: kenkeiter).
[Tuesday 14 September 2010] [22:57:34] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 14 September 2010] [23:27:41] Quit
keffo_ has left this server (Ping timeout: 240 seconds).
[Tuesday 14 September 2010] [23:43:31] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [00:03:41] Quit rgl has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [00:54:36] Join kenkeiter has joined this channel (
~kenkeiter@2002:4c73:685:1234:223:6cff:fe98:be3b).
[Wednesday 15 September 2010] [00:54:47] Quit kenkeiter has left this server (Client Quit).
[Wednesday 15 September 2010] [01:16:01] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 15 September 2010] [01:17:36] Quit guido_g has left this server (Client Quit).
[Wednesday 15 September 2010] [01:32:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [02:23:26] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [02:24:37] Quit bgranger has left this server (Client Quit).
[Wednesday 15 September 2010] [02:26:58] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 15 September 2010] [02:31:02] Quit keffo has left this server (Ping timeout: 272 seconds).
[Wednesday 15 September 2010] [02:47:24] Quit carrus85 has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [03:02:03] Quit kleppari has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [03:04:21] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [03:10:47] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Wednesday 15 September 2010] [03:15:41] Quit
AndrewBC_ has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [03:22:45] Quit plq has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [03:32:39] Quit rgl has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [04:06:06] <sustrik> Samy: pong
[Wednesday 15 September 2010] [04:06:38] <Zao> Doesn’t PGM stuff travel on UDP already?
[Wednesday 15 September 2010] [04:06:48] * Zao notes the six hour gap in time.
[Wednesday 15 September 2010] [04:09:05] <sustrik> Zao: PGM travels on IP
[Wednesday 15 September 2010] [04:09:22] <sustrik> epgm is a convenience hack that travels on UDP instead of IP
[Wednesday 15 September 2010] [04:09:32] <Zao> Oh, right.
[Wednesday 15 September 2010] [04:09:43] <Zao> Never used any of them, so I went with faint memories of manpages past.
[Wednesday 15 September 2010] [04:11:43] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Wednesday 15 September 2010] [04:53:30] Quit manveru has left this server (Quit: WeeChat 0.3.2).
[Wednesday 15 September 2010] [05:15:54] Join omarkj has joined this channel (
~omarkj@81.15.34.139).
[Wednesday 15 September 2010] [05:18:46] <omarkj> kleppari: Morning.
[Wednesday 15 September 2010] [05:28:58] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [05:29:15] <mrm2m> Hey ho!
[Wednesday 15 September 2010] [05:30:25] <mrm2m> Is there a lightweight introduction, how zmq works internally?
[Wednesday 15 September 2010] [05:33:01] <pieterh> mrm2m: there is documentation of the source code and there are whitepapers
[Wednesday 15 September 2010] [05:33:11] <pieterh> but there is nothing lightweight about 0MQ’s internals, really
[Wednesday 15 September 2010] [05:34:06] <mrm2m> ok
[Wednesday 15 September 2010] [05:39:21] <Bruc> hey wat sup all
[Wednesday 15 September 2010] [05:39:51] Part Bruc has left this channel.
[Wednesday 15 September 2010] [05:54:50] Quit xrfang has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [06:00:48] <kleppari> omarkj: hah, hi!
[Wednesday 15 September 2010] [06:01:15] <omarkj> kleppari: Didn’t notice you here until now, are you guys using 0mq?
[Wednesday 15 September 2010] [06:01:37] <kleppari> I’m experimenting a bit
[Wednesday 15 September 2010] [06:02:23] <kleppari> I just love the idea of message passing for concurrency
[Wednesday 15 September 2010] [06:03:08] <kleppari> well, outside of erlang at least
[Wednesday 15 September 2010] [06:05:14] <kleppari> how about yourself?
[Wednesday 15 September 2010] [06:07:23] <omarkj> Working with it.
[Wednesday 15 September 2010] [06:07:32] <omarkj> Let’s go pm. :)
[Wednesday 15 September 2010] [06:35:00] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [06:54:46] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [07:21:07] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [08:05:56] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [08:29:01] Quit jsimmons has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [08:49:08] <keffo> sustrik, I think I might have figured out the concurrency issue
[Wednesday 15 September 2010] [08:49:38] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [08:49:44] <sustrik> cremes: here i am
[Wednesday 15 September 2010] [08:50:00] <sustrik> keffo: so what’s the problem?
[Wednesday 15 September 2010] [08:51:59] <keffo> I was rewriting (using poll rather than sequential) and to clean up logging etc, and I just noticed one place where I send a null message before sending the rest, except “the rest” already included the incomin route-null, so I suspect it was being sent twie
[Wednesday 15 September 2010] [08:51:59] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [08:52:00] <keffo> twice
[Wednesday 15 September 2010] [08:52:40] <keffo> nullmore, nullmore, uuidmore, data, etc..
[Wednesday 15 September 2010] [08:52:52] <sustrik> ok, i see
[Wednesday 15 September 2010] [08:53:05] <sustrik> so no problem in 0mq to worry about
[Wednesday 15 September 2010] [08:53:10] <keffo> Not at all
[Wednesday 15 September 2010] [08:53:15] <sustrik> :)
[Wednesday 15 September 2010] [08:53:22] <keffo> I never expected that either :)
[Wednesday 15 September 2010] [08:53:48] <sustrik> you are an optimist then :)
[Wednesday 15 September 2010] [08:54:23] <keffo> It’s been very stable and well behaved so far, apart from this issue which is my own fault :)
[Wednesday 15 September 2010] [08:55:46] <keffo> I did 100k pi calcs yesterday, with 10 worker procs on different machines.. the loadbalancer had ~500kb/s passing through it, and in total more than half a gig of logs were generated.. That’s when I decided the logging was way wonky, didnt really tell much
[Wednesday 15 September 2010] [09:01:49] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 15 September 2010] [09:25:04] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [09:25:41] <cremes> sustrik: i’m available for the next hour or so if you want to chat
[Wednesday 15 September 2010] [09:32:23] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [09:45:14] <sustrik> cremes: hi
[Wednesday 15 September 2010] [09:46:04] <sustrik> give me some context first: why have you moved from ruby-ffi to rbzmq?
[Wednesday 15 September 2010] [09:46:49] <cremes> there are multiple ruby runtimes; not all of them support the C extension api that rbzmq uses
[Wednesday 15 September 2010] [09:47:00] <cremes> ffi allows me to support all of the runtimes
[Wednesday 15 September 2010] [09:47:38] <cremes> oh, and i haven’t moved from one to the other; i would just like rbzmq to have the same api as ffi-rzmq
[Wednesday 15 September 2010] [09:48:13] <sustrik> ok, i see
[Wednesday 15 September 2010] [09:48:27] <sustrik> it looks like brian buchanan is not reponding, right?
[Wednesday 15 September 2010] [09:48:39] <cremes> right
[Wednesday 15 September 2010] [09:48:48] <sustrik> ok, i have admin access to the project
[Wednesday 15 September 2010] [09:49:02] <sustrik> i’ll add you there as a developer
[Wednesday 15 September 2010] [09:49:10] <mato> sustrik: cremes: a question; i was wondering myself why there were two ruby bindings
[Wednesday 15 September 2010] [09:49:16] <mato> is there any point in keeping both around?
[Wednesday 15 September 2010] [09:49:20] <sustrik> hi
[Wednesday 15 September 2010] [09:49:37] <mato> if FFI is the way to go then why keep rbzmq at all?
[Wednesday 15 September 2010] [09:50:38] <cremes> um... i don’t know; it’s probably best to get feedback from folks using rbzmq and ask them why they prefer it
[Wednesday 15 September 2010] [09:50:53] <cremes> i’m biased towards the ffi one, obviously ;)
[Wednesday 15 September 2010] [09:51:01] <mato> that might be a good idea; having two bindings is IMO confusing
[Wednesday 15 September 2010] [09:51:03] <sustrik> does it work with any runtime?
[Wednesday 15 September 2010] [09:51:14] <cremes> that’s why i was hoping brian buchanan would speak up
[Wednesday 15 September 2010] [09:51:43] <mato> cremes: is there a mailing list for this? I’ve not seen any discussion on zeromq-dev...
[Wednesday 15 September 2010] [09:52:18] <mato> maybe just ping there, outline the situation (two bindings, ffi works on all runtimes, etc.) and ask the community
[Wednesday 15 September 2010] [09:52:21] <cremes> this has not been brought up on the zerome-dev list
[Wednesday 15 September 2010] [09:52:24] Quit RunningMike has left this server (Quit: ChatZilla 0.9.86 [Firefox 3.6.9/20100824153629]).
[Wednesday 15 September 2010] [09:52:39] <cremes> i’ll do that
[Wednesday 15 September 2010] [09:53:01] <sustrik> can you explain how does the runtime interoperability work?
[Wednesday 15 September 2010] [09:53:24] <sustrik> so far i had an impression that ffi doesn’t work with all the ruby runtimes
[Wednesday 15 September 2010] [09:53:43] <cremes> sustrik: as of a few days ago, ffi works with all of the runtimes
[Wednesday 15 September 2010] [09:53:48] <sustrik> aha
[Wednesday 15 September 2010] [09:53:50] <cremes> i worked out the last details with the runtime guys
[Wednesday 15 September 2010] [09:53:57] <sustrik> so it’s new
[Wednesday 15 September 2010] [09:54:09] <cremes> the issue with C extensions is that they oftentimes access lots of internal memory structures
[Wednesday 15 September 2010] [09:54:09] <sustrik> that may actually mean that rbzmq is not needed any more
[Wednesday 15 September 2010] [09:54:57] <mato> cremes: what’s the status of FFI support for Ruby MRI (i.e. that which most people are using right now)?
[Wednesday 15 September 2010] [09:54:59] <cremes> ffi allows the runtime authors to hide their implementation details and provide a more solid mechanism for accessing C libraries
[Wednesday 15 September 2010] [09:55:06] <sustrik> i see
[Wednesday 15 September 2010] [09:55:08] <cremes> mato: it’s quite good
[Wednesday 15 September 2010] [09:55:26] <mato> cremes: right, but does “quite good” mean “production quality”?
[Wednesday 15 September 2010] [09:55:57] <mato> i guess what i’m asking is if rbzmq goes away then what will that mean for the current mainstream of Ruby MRI users
[Wednesday 15 September 2010] [09:56:11] <mato> will they all just happily start using the FFI binding, or not?
[Wednesday 15 September 2010] [09:56:12] <cremes> mato: perhaps; i can only speak for how it works with 0mq; obviously we still have hangs until 2.1.x comes out with EINTR support for blocking calls
[Wednesday 15 September 2010] [09:56:39] <mato> sure, but the EINTR hangs are orthogonal to FFI/non-FFI
[Wednesday 15 September 2010] [09:56:41] <sustrik> well, whatever happens the rbzmq project is not going to be killed
[Wednesday 15 September 2010] [09:56:44] <sustrik> so no worry
[Wednesday 15 September 2010] [09:56:46] <cremes> mato: i’ll ask for feedback on the ML
[Wednesday 15 September 2010] [09:57:04] <sustrik> the problem is that it doesn’t have a permanent maintainer
[Wednesday 15 September 2010] [09:57:18] <cremes> mato: sure, it’s orthogonal, but it’s hard to say if there are other bugs when you get a hang
[Wednesday 15 September 2010] [09:57:25] <cremes> is it due to the blocking behavior or something else?
[Wednesday 15 September 2010] [09:57:31] <mato> well then put a note up about that (lack of maintainer) on the rbzmq project page
[Wednesday 15 September 2010] [09:57:39] <cremes> i don’t have the time or energy to run stuff under gdb all of the time to figure that out :)
[Wednesday 15 September 2010] [09:58:07] <cremes> let’s take this to the ML and see what users have to say
[Wednesday 15 September 2010] [09:58:10] <mato> yup
[Wednesday 15 September 2010] [09:58:12] <sustrik> ok
[Wednesday 15 September 2010] [09:58:18] <cremes> btw, from a search on github most projects are using the ffi-based binding
[Wednesday 15 September 2010] [09:58:36] <cremes> but i certainly don’t want to lose or alienate the rbzmq users
[Wednesday 15 September 2010] [09:59:01] <cremes> which takes us full circle; i’d like each one to present the same api to the user (based off of the C and C++ binding apis)
[Wednesday 15 September 2010] [09:59:11] <mato> definitely
[Wednesday 15 September 2010] [09:59:22] <mato> otherwise it just all gets way too confusing
[Wednesday 15 September 2010] [09:59:26] <cremes> if people can switch back and forth with no code changes, that’s a great situation for the community
[Wednesday 15 September 2010] [09:59:33] <mato> is it much work to synchronize the APIs?
[Wednesday 15 September 2010] [10:00:02] <cremes> mato: not a lot but unfortunately i don’t have the C chops to do it myself
[Wednesday 15 September 2010] [10:00:12] <cremes> so i need someone else to agree to it
[Wednesday 15 September 2010] [10:00:17] <cremes> to do the work
[Wednesday 15 September 2010] [10:00:42] <mato> cremes: well, ask for help on the list and we’ll see what happens
[Wednesday 15 September 2010] [10:00:53] <cremes> will do
[Wednesday 15 September 2010] [10:01:08] Quit pieterh has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [10:05:30] <sustrik> mato: man bug!
[Wednesday 15 September 2010] [10:05:44] <sustrik> zmq_socket(3):
[Wednesday 15 September 2010] [10:05:47] <sustrik> “When a ZMQ_XREQ socket is connected to a ZMQ_REP socket each message sent must consist of an empty message part, the delimiter, followed by one or more body parts.”
[Wednesday 15 September 2010] [10:05:48] <mato> sustrik: !!?!? :-)
[Wednesday 15 September 2010] [10:06:22] <sustrik> the identities should be mentioned
[Wednesday 15 September 2010] [10:06:48] <sustrik> they are definitely mentioned for ZMQ_XREP
[Wednesday 15 September 2010] [10:06:52] <ptrb> yeah, it’s not really clear to me what work i have to do at the app level to do XREQ/REP (or REQ/XREP)
[Wednesday 15 September 2010] [10:07:09] <mato> uh, yeah, i kind of punted on explaining the identity stack on the XREQ side at the time
[Wednesday 15 September 2010] [10:07:20] <mato> sustrik: that stuff was written in a hurry, you remember :)
[Wednesday 15 September 2010] [10:07:23] <sustrik> ptrb: there was a nice diagram somewhere...
[Wednesday 15 September 2010] [10:07:39] <mato> sustrik: anyway, it’s not a bug, it’s just a simplification :)
[Wednesday 15 September 2010] [10:07:43] <mato> sustrik: i’ll look into it.
[Wednesday 15 September 2010] [10:08:39] <cremes> if not, tell me what is unclear and i’ll fix it (or edit the page yourself)
[Wednesday 15 September 2010] [10:08:40] <ptrb> clicks
[Wednesday 15 September 2010] [10:08:52] <sustrik> ah – that’s the one i meant
[Wednesday 15 September 2010] [10:08:55] <cremes> i was certainly confused by that stuff which is why i wrote it down
[Wednesday 15 September 2010] [10:09:14] <ptrb> yeah, it definitely helps
[Wednesday 15 September 2010] [10:09:23] <ptrb> but ruby is definitely not the right language to use in examples like this
[Wednesday 15 September 2010] [10:09:47] <ptrb> it’s as opaque as perl without the benefit of perl’s saturation
[Wednesday 15 September 2010] [10:09:53] <cremes> ha
[Wednesday 15 September 2010] [10:10:03] <mato> :-)
[Wednesday 15 September 2010] [10:10:37] * mato will think of some suitably terse way of describing the identity stack in zmq_socket(3)
[Wednesday 15 September 2010] [10:36:52] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 15 September 2010] [10:45:23] <CIA-20> zeromq2: 03Martin Sustrik 07maint * re2802d9 10/ src/options.cpp : values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9lAWvv[Wednesday 15 September 2010] [10:46:54] <pieterh> sustrik: nice to see the validation of option values
[Wednesday 15 September 2010] [10:47:45] <sustrik> there are 3 bugs filled by cremes :)
[Wednesday 15 September 2010] [10:47:53] <mato> sustrik: incidentally, there’s one annoying thing regarding the option values
[Wednesday 15 September 2010] [10:48:00] <sustrik> yes?
[Wednesday 15 September 2010] [10:48:14] <mato> well, it’s the whole “option types” issue
[Wednesday 15 September 2010] [10:48:24] <sustrik> ?
[Wednesday 15 September 2010] [10:48:26] <mato> one thing that i realised the other day when i was writing some simple test cases
[Wednesday 15 September 2010] [10:48:47] <mato> is that in order to use options sensibly from C, you have to have uint64_t / int64_t defined
[Wednesday 15 September 2010] [10:49:03] <mato> now, obviously those are not defined by zmq.h
[Wednesday 15 September 2010] [10:49:25] <mato> of course if you want to use the API you have to do “platform specific stuff” to get those types
[Wednesday 15 September 2010] [10:49:31] <mato> not a good situation :-(
[Wednesday 15 September 2010] [10:50:27] <mato> especially on windows where M$ does not ship stdint.h
[Wednesday 15 September 2010] [10:50:36] <sustrik> i know
[Wednesday 15 September 2010] [10:50:54] <sustrik> what can i do without breaking the backward compatibility?
[Wednesday 15 September 2010] [10:51:15] <mato> probably nothing
[Wednesday 15 September 2010] [10:51:20] <sustrik> :|
[Wednesday 15 September 2010] [10:51:54] <mato> hmm, maybe...
[Wednesday 15 September 2010] [10:52:37] <mato> well, in theory you could define
zmq_ namespaced equivalents of those types including the platform magic in zmq.h, but that’s kind of ugly
[Wednesday 15 September 2010] [10:52:52] <mato> and it’d probably still break C++ due to it’s strict notions of types
[Wednesday 15 September 2010] [10:52:56] <sustrik> it has to be solved properly
[Wednesday 15 September 2010] [10:53:04] <sustrik> BSD-style
[Wednesday 15 September 2010] [10:53:13] <mato> hmm
[Wednesday 15 September 2010] [10:53:17] <sustrik> 4-byte unsinged integer in network byte order
[Wednesday 15 September 2010] [10:53:20] <mato> hang on
[Wednesday 15 September 2010] [10:53:21] <sustrik> and such
[Wednesday 15 September 2010] [10:53:22] <mato> what?
[Wednesday 15 September 2010] [10:53:30] <mato> what does network byte order have to do with it? :)
[Wednesday 15 September 2010] [10:53:45] <sustrik> that’s what i do when usign BSD sockets
[Wednesday 15 September 2010] [10:53:52] <mato> ?
[Wednesday 15 September 2010] [10:54:10] <sustrik> addr.port = htons (port);
[Wednesday 15 September 2010] [10:55:56] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [10:58:19] <mato> sustrik: ah, you’re right, i didn’t realise e.g. sockaddr_in.sin_port was in network byte order
[Wednesday 15 September 2010] [10:58:32] <mato> sustrik: the thing is, the setsockopt stuff is somewhat ad-hoc
[Wednesday 15 September 2010] [10:58:43] <CIA-20> zeromq2: 03Martin Sustrik 07master * re2802d9 10/ src/options.cpp : values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9lAWvv[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: 03Martin Sustrik 07master * rff10807 10/ src/options.cpp :
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: Merge branch ‘maint’
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: * maint:
[Wednesday 15 September 2010] [10:58:44] <CIA-20> zeromq2: values of RATE, RECOVERY_IVL and SWAP options are checked for negative values -
http://bit.ly/9P99Fu[Wednesday 15 September 2010] [10:58:48] <mato> it does not use network byte order AFAIK
[Wednesday 15 September 2010] [10:59:18] <mato> sustrik: anyway thinking about it, i guess zmq.h will have to define its own types
[Wednesday 15 September 2010] [11:00:14] <mato> the main problem with that is doing it in a portable fashion :-(
[Wednesday 15 September 2010] [11:07:22] <ptrb> #defines, #defines for everybody!!
[Wednesday 15 September 2010] [11:07:58] <mato> sure, feel free to suggest how to portably define a 64-bit unsigned integer type :-)
[Wednesday 15 September 2010] [11:08:20] <ptrb> you just take two ints and mash ‘em together, obviously
[Wednesday 15 September 2010] [11:08:27] <mato> :-)
[Wednesday 15 September 2010] [11:29:09] <sustrik> mato: btw, there’s some packaging PATCH on the mailing list
[Wednesday 15 September 2010] [11:29:15] <sustrik> will you reply to that?
[Wednesday 15 September 2010] [11:30:30] <mato> sustrik: yes, later, i know it’s there
[Wednesday 15 September 2010] [11:30:32] <mato> have to go now
[Wednesday 15 September 2010] [11:30:45] <sustrik> ok, there’s on in the bug tracker as well
[Wednesday 15 September 2010] [11:30:51] <mato> sustrik: i’d have just applied it but the license stuff needs to be sorted out
[Wednesday 15 September 2010] [11:31:05] <mato> else all i can reply is “please state your patch is licensed under...”
[Wednesday 15 September 2010] [11:31:14] <mato> have discussed with pieter will update you this evening
[Wednesday 15 September 2010] [11:31:18] <mato> or pieter will
[Wednesday 15 September 2010] [11:31:19] <mato> bbl
[Wednesday 15 September 2010] [11:31:32] <sustrik> should i ask for the license?
[Wednesday 15 September 2010] [11:31:46] <mato> don’t bother lets just fix it
[Wednesday 15 September 2010] [11:31:51] <mato> talk to you in the evening
[Wednesday 15 September 2010] [11:31:54] <sustrik> ok
[Wednesday 15 September 2010] [11:40:49] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 15 September 2010] [11:41:58] <Samy> sustrik, just had some questions regarding zeromq’s lock-less data structures.
[Wednesday 15 September 2010] [11:42:27] <sustrik> yes?
[Wednesday 15 September 2010] [11:42:51] <Samy> sustrik, I’m working on a library to help ease concurrent programming, it includes a plethora of concurrent data structures, synchronization methods and mechanisms for SMR (currently using hazard pointers, considering implementing RCU).
[Wednesday 15 September 2010] [11:43:37] <Samy> sustrik, I was curious if there were some features you would really like to see, and some specific constrainted data structures (M:N consumer/producers) for zeromq.
[Wednesday 15 September 2010] [11:43:52] <Samy> sustrik, I was also curious if SMR was important for ZeroMQ at all.
[Wednesday 15 September 2010] [11:44:38] <sustrik> sorry, what’s SMR?
[Wednesday 15 September 2010] [11:44:48] <Samy> sustrik, safe memory reclamation.
[Wednesday 15 September 2010] [11:45:26] <Samy> sustrik, usually it’s important for unbunded lock-less data structures (simple stack being a great example).
[Wednesday 15 September 2010] [11:46:48] <sustrik> does it translate to “make the current state of the memory visible to the other CPU codes”?
[Wednesday 15 September 2010] [11:47:23] <sustrik> sorry, i am not an expert on lock-free algos
[Wednesday 15 September 2010] [11:48:03] <Samy> sustrik, well, I guess fundamentally, it’s more like “let other CPUs know I am not using this memory”.
[Wednesday 15 September 2010] [11:48:27] <sustrik> ah, ok, let me explain what 0mq is doing
[Wednesday 15 September 2010] [11:48:28] <Samy> sustrik, some techniques will require a full barrier (RCU) while others do not (hazard pointers).
[Wednesday 15 September 2010] [11:48:41] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [11:48:57] <sustrik> the communication always happens between exactly two endpoins
[Wednesday 15 September 2010] [11:49:03] <sustrik> ie. at most 2 cpu cores
[Wednesday 15 September 2010] [11:49:25] <sustrik> so each lock-free queue has exactly one writer and exactly one reader
[Wednesday 15 September 2010] [11:49:30] <Samy> Ah, makes it much easier. :-)
[Wednesday 15 September 2010] [11:49:36] <Samy> Sorry, let me pop open the source-code.
[Wednesday 15 September 2010] [11:49:48] <sustrik> src/ypipe.hpp
[Wednesday 15 September 2010] [11:49:52] <Samy> Thanks, loading.
[Wednesday 15 September 2010] [11:50:26] <sustrik> what it does basically is that writer is appending new items to the linked list
[Wednesday 15 September 2010] [11:50:39] <sustrik> the reader is reading items from the linked list
[Wednesday 15 September 2010] [11:50:54] <Samy> I don’t see any padding, sustrik.
[Wednesday 15 September 2010] [11:51:08] <sustrik> what padding?
[Wednesday 15 September 2010] [11:51:09] <Samy> Between w and r.
[Wednesday 15 September 2010] [11:51:25] <Samy> sustrik, to prevent false cache line sharing, this improves concurrency.
[Wednesday 15 September 2010] [11:51:36] <sustrik> that would be nice
[Wednesday 15 September 2010] [11:51:49] <Samy> sustrik, it can be drastic. :-)
[Wednesday 15 September 2010] [11:51:53] <Samy> Let me show you a simple example on this machine.
[Wednesday 15 September 2010] [11:52:57] <sustrik> ok, so what has to be done is separate the variables that belong to the reader and those that belong to the writer
[Wednesday 15 September 2010] [11:53:04] <sustrik> and keep them in different cachelines
[Wednesday 15 September 2010] [11:53:06] <sustrik> right?
[Wednesday 15 September 2010] [11:53:27] <keffo> the data too
[Wednesday 15 September 2010] [11:53:48] <Samy> [
sbahra@sbahra validate]$ time ./ck_fifo_spsc 8 1 100000
[Wednesday 15 September 2010] [11:53:49] <Samy> real 0m5.294s
[Wednesday 15 September 2010] [11:53:59] <Samy> [
sbahra@sbahra validate]$ time ./ck_fifo_spsc 8 1 100000
[Wednesday 15 September 2010] [11:53:59] <Samy> real 0m7.718s
[Wednesday 15 September 2010] [11:54:12] <Samy> sustrik, the latter is without appropriate padding.
[Wednesday 15 September 2010] [11:54:16] <sustrik> nice
[Wednesday 15 September 2010] [11:54:26] <sustrik> the data are allocated in a contiguous block
[Wednesday 15 September 2010] [11:54:37] <keffo> that was pretty substantial, what arch is that?
[Wednesday 15 September 2010] [11:54:54] <sustrik> to avoid excessive memory allocation
[Wednesday 15 September 2010] [11:54:59] <Samy> sustrik, that is a simple benchmark that creates a token ring using a single-producer/single-consumer lock-less queue (passes around 100000 across 8 threads some number of iterations)
[Wednesday 15 September 2010] [11:55:05] <Samy> keffo, that is on a Nehalem box.
[Wednesday 15 September 2010] [11:55:09] <keffo> aho
[Wednesday 15 September 2010] [11:55:25] <keffo> sustrik, You should look into pooling, not only for that
[Wednesday 15 September 2010] [11:55:40] <Samy> sustrik, you want the writer and reader variables to be on a seperate cache line.
[Wednesday 15 September 2010] [11:55:56] <sustrik> right, i understand that
[Wednesday 15 September 2010] [11:55:56] <Samy> sustrik, so for example, void
*reader; char pad[56]; void
*writer; ...
[Wednesday 15 September 2010] [11:56:00] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [11:56:12] <sustrik> not sure how to do that in portable fashion though
[Wednesday 15 September 2010] [11:56:22] <keffo> compile time :)
[Wednesday 15 September 2010] [11:56:28] <keffo> macro hell
[Wednesday 15 September 2010] [11:56:29] <ptrb> did someone say #define??
[Wednesday 15 September 2010] [11:56:32] <ptrb> :D
[Wednesday 15 September 2010] [11:56:34] <Samy> sustrik, not very portable (my library tries to make that portable by doing the hard work of generating those constants at compile-time).
[Wednesday 15 September 2010] [11:56:34] <sustrik> :)
[Wednesday 15 September 2010] [11:56:49] <Samy> sustrik, but modern IA32 and SPARCv9 boxes I know of all have 64 byte cache lines.
[Wednesday 15 September 2010] [11:56:57] <sustrik> well, if we assumed the cache line size is 64 bytes
[Wednesday 15 September 2010] [11:56:59] <keffo> cpuid fiddling works I guess :)
[Wednesday 15 September 2010] [11:57:03] <sustrik> we would be right in most cases
[Wednesday 15 September 2010] [11:57:06] <sustrik> no?
[Wednesday 15 September 2010] [11:57:14] <keffo> all x86_64 is >64 no?
[Wednesday 15 September 2010] [11:57:21] <sustrik> 128?
[Wednesday 15 September 2010] [11:57:22] <Samy> keffo, no.
[Wednesday 15 September 2010] [11:57:25] <Samy> keffo, 64.
[Wednesday 15 September 2010] [11:57:38] <Samy> keffo, when I mean IA32, I mean x86/x86_64.
[Wednesday 15 September 2010] [11:57:41] <keffo> >=64 then :)
[Wednesday 15 September 2010] [11:57:47] <Samy> I don’t know of any > 64.
[Wednesday 15 September 2010] [11:58:18] <sustrik> padding to 64 bytes seems reasonable imo
[Wednesday 15 September 2010] [11:58:34] <keffo> I meant, x86_64 is presumably >=64, while x86 is not
[Wednesday 15 September 2010] [11:58:50] <sustrik> that would be 64 bytes per reader and 64 bytes per writer
[Wednesday 15 September 2010] [11:59:06] <sustrik> the shared varaible should probably be on a separate cahceline
[Wednesday 15 September 2010] [11:59:08] <keffo> Samy, Have you tried measuring on itanium? :)
[Wednesday 15 September 2010] [11:59:16] <sustrik> so it’s 192 bytes per queue
[Wednesday 15 September 2010] [11:59:40] <Samy> keffo, Itanium is very interesting, but barely used, so no.
[Wednesday 15 September 2010] [12:00:06] <Samy> keffo, I would like to work on a port of my library there but I have had design issues, just recently did a major design change.
[Wednesday 15 September 2010] [12:00:07] <keffo> Yeah, void of any cachemisses it should be
[Wednesday 15 September 2010] [12:00:20] <Samy> sustrik, sorry, can you continue with your explanation of ypipe?
[Wednesday 15 September 2010] [12:00:25] <sustrik> ah
[Wednesday 15 September 2010] [12:00:42] * keffo butts out :)
[Wednesday 15 September 2010] [12:00:42] <sustrik> ok, so part of the list belongs to the writer thread
[Wednesday 15 September 2010] [12:00:50] <sustrik> part of it to the reader thread
[Wednesday 15 September 2010] [12:01:13] <sustrik> the only point where the two interact is when reader has no more data to read
[Wednesday 15 September 2010] [12:01:21] <sustrik> then it does CAS
[Wednesday 15 September 2010] [12:01:33] <sustrik> to get the writer’s portion of the list
[Wednesday 15 September 2010] [12:02:01] <Samy> So, basically, you batch the operations?
[Wednesday 15 September 2010] [12:02:06] <sustrik> exactly
[Wednesday 15 September 2010] [12:02:09] <sustrik> that’s the key
[Wednesday 15 September 2010] [12:02:14] <Samy> So, writer does enqueue operations to the queue.
[Wednesday 15 September 2010] [12:02:20] <sustrik> ack
[Wednesday 15 September 2010] [12:02:21] <Samy> The reader occassionally does a batch dequeue.
[Wednesday 15 September 2010] [12:02:24] <sustrik> ack
[Wednesday 15 September 2010] [12:02:27] <Samy> Ok.
[Wednesday 15 September 2010] [12:02:48] <Samy> sustrik, why do you use CAS for the batch dequeue? That can be implemented using an xchg, making the batch dequeue a wait-free operation.
[Wednesday 15 September 2010] [12:02:48] <sustrik> it’s pretty effective just because of the batching
[Wednesday 15 September 2010] [12:03:24] <sustrik> hm, i haven’t seen the code for a long time
[Wednesday 15 September 2010] [12:03:27] <sustrik> let me have a look
[Wednesday 15 September 2010] [12:04:02] <Samy> It looks like you do a single CAS.
[Wednesday 15 September 2010] [12:04:12] <Samy> Line 137, check_read (I assume?)
[Wednesday 15 September 2010] [12:04:56] * Samy looks at yqueue
[Wednesday 15 September 2010] [12:05:01] <sustrik> “If there are no
[Wednesday 15 September 2010] [12:05:01] <sustrik> // items to prefetch, set c to NULL”
[Wednesday 15 September 2010] [12:05:09] <sustrik> Samy: yes
[Wednesday 15 September 2010] [12:05:27] <sustrik> the cas is used to communicate back to the writer
[Wednesday 15 September 2010] [12:05:28] <Samy> You can implement this without any lock-free operations.
[Wednesday 15 September 2010] [12:05:43] * sustrik is listening
[Wednesday 15 September 2010] [12:06:32] <sustrik> you mean without atomic ops?
[Wednesday 15 September 2010] [12:06:41] <sustrik> or bus locking?
[Wednesday 15 September 2010] [12:07:14] <Samy> Without explicit atomic operations.
[Wednesday 15 September 2010] [12:07:40] <Samy> sustrik, the idea is to share a stub node for both tail and head of the queue.
[Wednesday 15 September 2010] [12:08:16] <sustrik> can you explain in more detail?
[Wednesday 15 September 2010] [12:08:17] <Samy> sustrik, writer updates tail, always updating the next pointer, and reader always assume first node is stub entry.
[Wednesday 15 September 2010] [12:08:31] <Samy> sustrik, I can show you source-code.
[Wednesday 15 September 2010] [12:08:39] <Samy> Let me see if I find it, hold on.
[Wednesday 15 September 2010] [12:09:02] <sustrik> ok
[Wednesday 15 September 2010] [12:09:35] <sustrik> how does the communication between two cpu cores happens then?
[Wednesday 15 September 2010] [12:09:41] <sustrik> cache coherency algos?
[Wednesday 15 September 2010] [12:11:21] <Samy> sustrik, yes.
[Wednesday 15 September 2010] [12:11:35] <Samy> sustrik, the point is, the only time we need to really share state is if the queue is empty.
[Wednesday 15 September 2010] [12:11:43] <sustrik> ack
[Wednesday 15 September 2010] [12:11:50] <Samy> sustrik, we need a way to detect if the queue is empty atomically and update both head and tail atomically.
[Wednesday 15 September 2010] [12:11:54] <Samy> sustrik, by sharing a stub node, this is possible.
[Wednesday 15 September 2010] [12:12:23] <sustrik> i think something like that is done in ypipe
[Wednesday 15 September 2010] [12:12:33] <sustrik> there’s an emty item in the list
[Wednesday 15 September 2010] [12:12:40] <sustrik> that servers as a placeholder
[Wednesday 15 September 2010] [12:12:50] <sustrik> that’s what you mean by stub, right?
[Wednesday 15 September 2010] [12:12:52] <Samy> Yes.
[Wednesday 15 September 2010] [12:13:20] <Samy> This isn’t fenced correctly, but that should be fine on IA32.
[Wednesday 15 September 2010] [12:14:08] <sustrik> i thought that IA requires you to fence explicitely
[Wednesday 15 September 2010] [12:14:23] <sustrik> ah, you mean x85
[Wednesday 15 September 2010] [12:14:25] <sustrik> 86
[Wednesday 15 September 2010] [12:14:57] <Samy> Yes.
[Wednesday 15 September 2010] [12:15:25] <Samy> The fences may be sufficient, I just haven’t verified it on non-IA32 yet. :-)
[Wednesday 15 September 2010] [12:15:25] <sustrik> ok, there’s one more issue there
[Wednesday 15 September 2010] [12:15:48] <sustrik> when reader finds out that there are no more items to read
[Wednesday 15 September 2010] [12:15:53] <sustrik> it goes asleep
[Wednesday 15 September 2010] [12:16:07] <sustrik> it’s writers responsibity to wake it up
[Wednesday 15 September 2010] [12:16:29] <sustrik> so the writer has to be informed about the fact that reader tried to get more items and failed
[Wednesday 15 September 2010] [12:16:41] <Samy> sustrik, futex(2) works well.
[Wednesday 15 September 2010] [12:16:49] <sustrik> yes, it’s similar
[Wednesday 15 September 2010] [12:17:10] <sustrik> but keep in mind that this is a multi-platform app
[Wednesday 15 September 2010] [12:17:47] <Samy> Things sort of suck without futex. :-(
[Wednesday 15 September 2010] [12:18:01] <Samy> (or a similar mechanism, at least in ring-3)
[Wednesday 15 September 2010] [12:18:08] <sustrik> that’s how it is :|
[Wednesday 15 September 2010] [12:18:28] <Samy> But you could abstract a generic CV layer that uses futex directly if available.
[Wednesday 15 September 2010] [12:18:40] <Samy> sustrik, where is this wake-up mechanism implemented?
[Wednesday 15 September 2010] [12:18:59] <sustrik> out of the class, it’s a dumb socket pair
[Wednesday 15 September 2010] [12:19:09] <sustrik> i had a futex implementation once
[Wednesday 15 September 2010] [12:19:15] <Samy> What happened?
[Wednesday 15 September 2010] [12:19:22] <sustrik> but it was pain in the ass to make it work everywhere
[Wednesday 15 September 2010] [12:19:28] <Samy> I see.
[Wednesday 15 September 2010] [12:19:32] <sustrik> some linux kernels pretend to have futexes
[Wednesday 15 September 2010] [12:19:46] <sustrik> but astucally return ENOTSUP when you try to use them
[Wednesday 15 September 2010] [12:19:47] <sustrik> and alike
[Wednesday 15 September 2010] [12:19:57] <Samy> I see.
[Wednesday 15 September 2010] [12:20:14] <sustrik> anyway, the wake up mechanism is irrelevant for this discussion
[Wednesday 15 September 2010] [12:20:31] <Samy> Mostly, yes.
[Wednesday 15 September 2010] [12:20:36] <sustrik> what’s relevant is the writer has to be notified that reader is sleeping
[Wednesday 15 September 2010] [12:20:46] <sustrik> that’s what the second cas is for
[Wednesday 15 September 2010] [12:20:55] <sustrik> in flush function
[Wednesday 15 September 2010] [12:21:17] <Samy> I don’t understand.
[Wednesday 15 September 2010] [12:21:30] <Samy> Why not simply use plain loads and stores if this is single reader and consumer?
[Wednesday 15 September 2010] [12:22:06] <sustrik> because one bit of information is passed the other way round
[Wednesday 15 September 2010] [12:22:10] <sustrik> from reader to writer
[Wednesday 15 September 2010] [12:22:19] <sustrik> “i am sleeping”
[Wednesday 15 September 2010] [12:22:28] <sustrik> that’s when c is set to NULL
[Wednesday 15 September 2010] [12:22:50] <sustrik> maybe it can be done without atomic ops, i am not an expert
[Wednesday 15 September 2010] [12:23:20] <sustrik> however, if it was a normal locking code, this would be a place where races can occur
[Wednesday 15 September 2010] [12:24:12] <Samy> Ok.
[Wednesday 15 September 2010] [12:24:40] <sustrik> so, reader tries to get the latest batch of the items
[Wednesday 15 September 2010] [12:24:51] <sustrik> and if there are none, it sets c to NULL
[Wednesday 15 September 2010] [12:25:29] <sustrik> in flush function, writer adds new batch of items and if it finds out that c was NULL previously
[Wednesday 15 September 2010] [12:25:35] <sustrik> it knows the reader is sleeping
[Wednesday 15 September 2010] [12:25:42] <sustrik> and that it should wake it up
[Wednesday 15 September 2010] [12:26:11] <sustrik> my feeling is that the operations on c have to be atomic
[Wednesday 15 September 2010] [12:26:18] <sustrik> not 100% sure though
[Wednesday 15 September 2010] [12:30:42] <Samy> read_asleep = false; loop { old = xchg(top, NULL); if old == null then reader_asleep = true; sleep(); }
[Wednesday 15 September 2010] [12:31:25] <Samy> Sorry, read_asleep is in loop.
[Wednesday 15 September 2010] [12:32:40] <sustrik> that’s the writer side?
[Wednesday 15 September 2010] [12:33:50] <Samy> Reader.
[Wednesday 15 September 2010] [12:34:09] <Samy> There is still a risk of spurious wake-up, but that isn’t a big deal.
[Wednesday 15 September 2010] [12:34:17] <Samy> If it is, then you basically implement a barrier.
[Wednesday 15 September 2010] [12:34:55] <Samy> You have a reader and a writer shared variable. Writer signals reader once, only if reader has signaled writer since last wake-up.
[Wednesday 15 September 2010] [12:35:55] <sustrik> yes, that’s the goal
[Wednesday 15 September 2010] [12:36:03] <Samy> And the algorithm.
[Wednesday 15 September 2010] [12:36:15] * sustrik is a dummy
[Wednesday 15 September 2010] [12:36:21] <Samy> Me too.
[Wednesday 15 September 2010] [12:36:28] <sustrik> can you give a pseudo-code for writer side as well?
[Wednesday 15 September 2010] [12:38:19] <sustrik> in the code above reader_asleep is another shared variable, right?
[Wednesday 15 September 2010] [12:39:32] <pieterh> sustrik: I was trying to say something about options but got interrupted by stuff...
[Wednesday 15 September 2010] [12:40:02] <pieterh> API should IMO reject an identity starting with zero byte
[Wednesday 15 September 2010] [12:40:19] <sustrik> now you are likely to get interrupted by brain-damaging lock-free algo discussion :)
[Wednesday 15 September 2010] [12:40:33] <pieterh> :-/
[Wednesday 15 September 2010] [12:40:34] <sustrik> yes, that would be nice
[Wednesday 15 September 2010] [12:41:04] <Samy> reader: loop { r = false; store_fence; w = true; o = xchg(top, NULL); if old == null then { r = true; sleep(); } }
[Wednesday 15 September 2010] [12:41:05] <Samy> writer: add(); c_w = w; load_fence; c_r = r; if c_r == true and c_w == true; then { c_w = false; signal_reader; }
[Wednesday 15 September 2010] [12:41:24] <sustrik> pieter: fancy sumbitting a patch?
[Wednesday 15 September 2010] [12:41:49] <pieterh> sustrik: I’m pulling master as we speak, yes I’ll make you a patch
[Wednesday 15 September 2010] [12:42:45] <sustrik> thanks
[Wednesday 15 September 2010] [12:42:56] <sustrik> Samy: sorry, having 3 discussions in parallel
[Wednesday 15 September 2010] [12:43:34] <Samy> Hey, no problem. :-)
[Wednesday 15 September 2010] [12:43:39] <Samy> I’ll be back in some minutes.
[Wednesday 15 September 2010] [12:43:53] <sustrik> sure, checking your code right now
[Wednesday 15 September 2010] [12:46:14] <sustrik> what’s top?
[Wednesday 15 September 2010] [12:46:50] <sustrik> which variable belong to whom?
[Wednesday 15 September 2010] [12:46:57] <sustrik> both r and w seem to be shared
[Wednesday 15 September 2010] [12:48:23] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 15 September 2010] [12:49:34] <pieterh> sustrik: np. options.cpp already checks that... :-)
[Wednesday 15 September 2010] [12:49:43] <sustrik> good
[Wednesday 15 September 2010] [12:51:56] <Samy> sustrik, yes, they are shared.
[Wednesday 15 September 2010] [12:52:46] <Samy> sustrik, that statement is whatever condition you use to check if the queue is empty.
[Wednesday 15 September 2010] [12:53:17] <sustrik> this one: o = xchg(top, NULL); ?
[Wednesday 15 September 2010] [12:53:25] <Samy> That and next.
[Wednesday 15 September 2010] [12:53:34] <Samy> o == null
[Wednesday 15 September 2010] [12:53:35] <Samy> (not old)
[Wednesday 15 September 2010] [12:54:11] <sustrik> do c_r and c_w are local to writer
[Wednesday 15 September 2010] [12:54:17] <Samy> Yes.
[Wednesday 15 September 2010] [12:54:32] * sustrik is trying to make sense of it
[Wednesday 15 September 2010] [12:54:58] <Samy> The reason we have 2 variables is that writer will signal the reader only once.
[Wednesday 15 September 2010] [12:55:40] <Samy> The reader will then wake-up, and if the queue is empty it will indicate that it is asleep and it is fine for the writer to send another signal.
[Wednesday 15 September 2010] [12:56:22] <Samy> sustrik, if you treat it like a stack, it will make more sense.
[Wednesday 15 September 2010] [12:56:50] <sustrik> Samy: stack of what?
[Wednesday 15 September 2010] [12:56:54] <Samy> sustrik, for a FIFO, you can use another technique.
[Wednesday 15 September 2010] [12:57:04] <Samy> sustrik, of objects.
[Wednesday 15 September 2010] [12:57:11] <sustrik> that’s the case in 0MQ
[Wednesday 15 September 2010] [12:57:20] <Samy> sustrik, this is in reference of xchg(top, ...) :)
[Wednesday 15 September 2010] [12:57:32] <Samy> sustrik, what’s of relevance to you is the notion of the r and w variables, that’s all.
[Wednesday 15 September 2010] [12:58:03] <sustrik> r = “i am sleeeping”
[Wednesday 15 September 2010] [12:58:14] <sustrik> what abour w?
[Wednesday 15 September 2010] [12:58:19] <sustrik> about*
[Wednesday 15 September 2010] [12:58:39] <Samy> w = “reader has woken-up since the last signal”
[Wednesday 15 September 2010] [12:59:20] <sustrik> soemging is missing
[Wednesday 15 September 2010] [12:59:25] <sustrik> w is never set to false
[Wednesday 15 September 2010] [12:59:34] <Samy> Oh, sorry.
[Wednesday 15 September 2010] [12:59:44] <Samy> { c_w = false; signal_reader; } should be { w = false; signal_reader; }
[Wednesday 15 September 2010] [13:00:14] <sustrik> lk,now it makes sense
[Wednesday 15 September 2010] [13:00:56] <sustrik> now, what about the fences
[Wednesday 15 September 2010] [13:01:29] <sustrik> does it sync the data in the queue as well?
[Wednesday 15 September 2010] [13:02:03] <Samy> You can figure that out.
[Wednesday 15 September 2010] [13:02:22] <Samy> That is mainly meant for those flags.
[Wednesday 15 September 2010] [13:02:29] Quit AndrewBC has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [13:02:56] <Samy> The idea, reader should always appear to be set to false before writer is set to true (it must indicate it has woken up before it indicates it is fine to send another signal).
[Wednesday 15 September 2010] [13:03:42] <sustrik> got it
[Wednesday 15 September 2010] [13:04:14] <sustrik> that part ensures that everything happens in nice step-lock fashion
[Wednesday 15 September 2010] [13:04:19] <Samy> Right.
[Wednesday 15 September 2010] [13:04:42] <sustrik> now few technical questions...
[Wednesday 15 September 2010] [13:04:49] <Samy> I’m no expert, but ok.
[Wednesday 15 September 2010] [13:05:02] <sustrik> what does lock;xchg() actually means
[Wednesday 15 September 2010] [13:05:11] <sustrik> is there an implied barrier?
[Wednesday 15 September 2010] [13:05:29] <Samy> On IA32, atomic operations have a total order across all processors.
[Wednesday 15 September 2010] [13:05:46] <Samy> There is an implicit pipeline flush.
[Wednesday 15 September 2010] [13:06:11] <Samy> There is an implied barrier.
[Wednesday 15 September 2010] [13:06:24] <Samy> There is also no need for the lock prefix on xchg.
[Wednesday 15 September 2010] [13:06:41] <Samy> xchg is guaranteed to be atomic (which is what also makes it expensive to use, if you don’t need it).
[Wednesday 15 September 2010] [13:06:42] <sustrik> in that case there’s no global ordering, right?
[Wednesday 15 September 2010] [13:06:52] <Samy> Usually, yes.
[Wednesday 15 September 2010] [13:07:10] <Samy> add is not guaranteed to be atomic, lock add is.
[Wednesday 15 September 2010] [13:07:16] <Samy> xchg is always atomic.
[Wednesday 15 September 2010] [13:07:21] <sustrik> oh my
[Wednesday 15 September 2010] [13:07:41] <Samy> It’s expensive compared to simple loads and stores, that’s for sure.
[Wednesday 15 September 2010] [13:07:41] <sustrik> so there’s an implied barrier on xchg, right?
[Wednesday 15 September 2010] [13:07:45] <Samy> Yes.
[Wednesday 15 September 2010] [13:07:58] <sustrik> then, in your code we have 2 barriers
[Wednesday 15 September 2010] [13:08:03] <Samy> For more information, you can see Volume 3 of the Intel Architecture Manuals, Chapter 8.
[Wednesday 15 September 2010] [13:08:37] <sustrik> isn’t it better to use single cas with a barrier then?
[Wednesday 15 September 2010] [13:08:38] <Samy> sustrik, you don’t need any of those barriers for IA32. Stores are always seen in order.
[Wednesday 15 September 2010] [13:09:19] <Samy> sustrik, CAS is expensive.
[Wednesday 15 September 2010] [13:09:31] <Samy> sustrik, it’s best to avoid the lock prefix completely if possible.
[Wednesday 15 September 2010] [13:09:37] <sustrik> ack
[Wednesday 15 September 2010] [13:09:41] <Samy> sustrik, atomic loads and stores do this.
[Wednesday 15 September 2010] [13:10:08] <sustrik> atomic load & stores == have global ordering ?
[Wednesday 15 September 2010] [13:10:12] <Samy> sustrik, in theory, CAS can provide infinite consensus. If you don’t need infinite consensus, you might not need CAS at all.
[Wednesday 15 September 2010] [13:10:50] <sustrik> anyway, i have to get my head around it
[Wednesday 15 September 2010] [13:11:00] <sustrik> are you around somewhere?
[Wednesday 15 September 2010] [13:11:01] <Samy> Processor ordering.
[Wednesday 15 September 2010] [13:11:10] <sustrik> email, or so?
[Wednesday 15 September 2010] [13:11:29] <Samy> sustrik, I’d love your feedback on this library I’m working on sometime.
[Wednesday 15 September 2010] [13:11:36] <Samy> sustrik, what architectures does ZeroMQ support?
[Wednesday 15 September 2010] [13:12:26] <sustrik> we’ve tested on x86, itanium, sparc, ppc
[Wednesday 15 September 2010] [13:12:28] <sustrik> arm
[Wednesday 15 September 2010] [13:12:29] <Samy> sustrik, if you see section 8.2.2 in Volume 3 of the IA32 manuals, you’ll get a nice break-down of the memory ordering.
[Wednesday 15 September 2010] [13:12:58] <Samy> sustrik, does the ZeroMQ project have regular access to such boxes?
[Wednesday 15 September 2010] [13:13:21] <sustrik> we have an itanium and sparc box
[Wednesday 15 September 2010] [13:13:28] <sustrik> no ppc
[Wednesday 15 September 2010] [13:13:28] <Samy> Itanium, nice. :-)
[Wednesday 15 September 2010] [13:13:38] <Samy> My PPC box died (Mac Mini) recently. Need to fix it.
[Wednesday 15 September 2010] [13:14:01] <Samy> I sold my only decent SPARCv9 box, at work our SPARCv9 machines are tied to our build cluster.
[Wednesday 15 September 2010] [13:14:21] <Samy> sustrik, if I could get accounts on those, I can port my library’s atomic interface for them to support those.
[Wednesday 15 September 2010] [13:14:31] <Samy> sustrik, might be useful, at least as a reference implementation of some structures.
[Wednesday 15 September 2010] [13:15:09] <Samy> sustrik, ;]
[Wednesday 15 September 2010] [13:15:46] <sustrik> our sparc box is extremely old and slow :)
[Wednesday 15 September 2010] [13:15:54] <Samy> Itanium is fine too.
[Wednesday 15 September 2010] [13:16:07] <Samy> Well, Itanium is what I really want access to.
[Wednesday 15 September 2010] [13:16:10] <sustrik> but mato mumbled something about bringing some 14-core sparc
[Wednesday 15 September 2010] [13:16:16] <sustrik> ah
[Wednesday 15 September 2010] [13:16:36] <sustrik> i have to ask mato, i think there’s some applications running on it
[Wednesday 15 September 2010] [13:16:42] <keffo> sandy bridge ftw!
[Wednesday 15 September 2010] [13:16:48] <Samy> sustrik, ok.
[Wednesday 15 September 2010] [13:17:20] <Samy> I look forward to future discussions. I would like to show you the work I’ve done so far for feedback.
[Wednesday 15 September 2010] [13:17:34] <Samy> It is C, not C++, however.
[Wednesday 15 September 2010] [13:17:41] <sustrik> Samy, i can have a look but as you see i am not an expert :)
[Wednesday 15 September 2010] [13:18:13] <Samy> Well, that’s fine. The idea is, what would it take to make you use this? As a 3rd party looking to integrate this into their product, what issues do you have with the interface?
[Wednesday 15 September 2010] [13:18:28] <Samy> etc
[Wednesday 15 September 2010] [13:18:35] <sustrik> what’s the link to the lib?
[Wednesday 15 September 2010] [13:18:43] <kleppari> we’re ditching a couple of sun t1000 boxes at work
[Wednesday 15 September 2010] [13:18:48] <Samy> No link. Tarball, sustrik.
[Wednesday 15 September 2010] [13:18:54] <Samy> When we have a discussion, I can share that.
[Wednesday 15 September 2010] [13:18:55] <kleppari> let me see if I can ‘rescue’ on of them
[Wednesday 15 September 2010] [13:19:07] <kleppari> s/on/one/
[Wednesday 15 September 2010] [13:19:19] Quit mrm2m has left this server (Quit: Leaving.).
[Wednesday 15 September 2010] [13:20:11] <Samy> Cool.
[Wednesday 15 September 2010] [13:20:16] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [13:20:19] <Samy> I’m back to work, take care.
[Wednesday 15 September 2010] [13:20:41] <sustrik> you too
[Wednesday 15 September 2010] [13:20:42] <sustrik> bye
[Wednesday 15 September 2010] [13:20:47] <sustrik> kleppari: i don’t think it’s really needed
[Wednesday 15 September 2010] [13:21:15] <sustrik> unless you need it yourself
[Wednesday 15 September 2010] [13:21:41] <kleppari> not really
[Wednesday 15 September 2010] [13:21:47] <kleppari> kind of lost interest in solaris after oracle bought it
[Wednesday 15 September 2010] [13:21:53] <kleppari> err, after oracle bought sun
[Wednesday 15 September 2010] [13:22:00] <kleppari> I think they’ll do a wonderful job killing it
[Wednesday 15 September 2010] [13:22:23] <sustrik> yeah, looks like there will be a lot of sun boxes avialable :)
[Wednesday 15 September 2010] [13:22:42] <kleppari> heh, yeah - red hat will make a fortune
[Wednesday 15 September 2010] [13:23:37] <kleppari> but the t1000 was a good box, I still think 16 concurrent threads of execution per core is impressive
[Wednesday 15 September 2010] [13:24:57] <kleppari> t2000, sorry
[Wednesday 15 September 2010] [13:27:14] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Wednesday 15 September 2010] [13:29:59] <pieterh> kleppari: we used to have a t2000 at iMatix when we were making OpenAMQ
[Wednesday 15 September 2010] [13:30:11] <pieterh> it was a pretty impressive box, made a noise like an aircraft taking off
[Wednesday 15 September 2010] [13:30:30] <pieterh> and was slower (all 32 cores or whatever) than a 2-core Athlon
[Wednesday 15 September 2010] [13:31:10] Quit ModusPwnens has left this server.
[Wednesday 15 September 2010] [13:31:10] <mato> pieterh: it still might be more useful than the old e4500 i can get for the project
[Wednesday 15 September 2010] [13:31:16] <kleppari> slower at what?
[Wednesday 15 September 2010] [13:31:22] <mato> pieterh: 0mq is a very different codebase from OpenAMQ
[Wednesday 15 September 2010] [13:32:01] <pieterh> mato: perhaps, yes
[Wednesday 15 September 2010] [13:32:12] <mato> kleppari: if you do get a chance to rescue one i think it’d be more interesting for development of the lockfree algorithms in 0mq than what i’ve been offered
[Wednesday 15 September 2010] [13:32:14] <pieterh> kleppari: at raw I/O afaics
[Wednesday 15 September 2010] [13:32:24] <pieterh> but the main difference was probably Linux vs. Solaris
[Wednesday 15 September 2010] [13:32:25] <mato> kleppari: which is an old E4500 (fully spec’ed with 14 CPUs)
[Wednesday 15 September 2010] [13:32:34] <kleppari> I found that these boxes perform pretty well with a huge runqueue
[Wednesday 15 September 2010] [13:32:42] <kleppari> mato: let me see what I can do
[Wednesday 15 September 2010] [13:32:50] <pieterh> they were designed for web services, indeed
[Wednesday 15 September 2010] [13:32:56] <kleppari> mato: can’t promise anything, though, but I’ll try
[Wednesday 15 September 2010] [13:32:58] Quit xla has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [13:33:10] <mato> kleppari: no hurry; also, where are you located? it’s not worth shipping that stuff from outside of the EU, too expensive
[Wednesday 15 September 2010] [13:33:18] <kleppari> iceland
[Wednesday 15 September 2010] [13:33:22] <mato> lol :)
[Wednesday 15 September 2010] [13:33:29] <kleppari> so the shipping might kill the deal :P
[Wednesday 15 September 2010] [13:33:42] <mato> kleppari: precisely, unless you have friends at smyril line :-)
[Wednesday 15 September 2010] [13:33:54] <mato> “chuck it down with them bananas” :-)
[Wednesday 15 September 2010] [13:34:12] <kleppari> heheh
[Wednesday 15 September 2010] [13:35:50] <kleppari> I don’t think we’d be shipping bananas out of the country, but fair point :P
[Wednesday 15 September 2010] [13:36:15] <pieterh> hey... ebay is offering free shipping or something...
[Wednesday 15 September 2010] [13:36:54] <mato> kleppari: thanks for the offer in any case
[Wednesday 15 September 2010] [13:36:58] * mato is off to 0beer :-)
[Wednesday 15 September 2010] [13:37:05] <pieterh> 0beer0clock
[Wednesday 15 September 2010] [13:37:08] <mato> beer
[Wednesday 15 September 2010] [13:37:12] <pieterh> zbeer
[Wednesday 15 September 2010] [13:37:13] <mato> cyl
[Wednesday 15 September 2010] [13:37:18] <pieterh> cyrsn
[Wednesday 15 September 2010] [13:38:10] * pieterh suspects mato has a macro that types
[Wednesday 15 September 2010] [13:38:22] <mato> !
[Wednesday 15 September 2010] [13:38:56] <pieterh> :-) I knew it...
[Wednesday 15 September 2010] [13:39:08] <pieterh> oh... hang on...
[Wednesday 15 September 2010] [13:39:20] Quit AndrewBC has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [13:39:45] * pieterh wonders how to config xmodmap to do this...
[Wednesday 15 September 2010] [13:43:42] <kleppari> alt gr + o ?
[Wednesday 15 September 2010] [13:43:45] <kleppari> works on the .is layout
[Wednesday 15 September 2010] [13:44:07] <guido_g> and on 105 key .de (qwertz)
[Wednesday 15 September 2010] [13:44:39] <guido_g> and I just saw that ch. 3 is back...
[Wednesday 15 September 2010] [13:44:45] <pieterh> yeah, I fixed it
[Wednesday 15 September 2010] [13:45:03] * guido_g is reading
[Wednesday 15 September 2010] [13:45:14] Quit pieterh has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [13:46:48] <pieterh> cyal, i’m off for zbeer
[Wednesday 15 September 2010] [13:47:19] <pieterh> guido_g: there’s also a new problem solver in Ch1
[Wednesday 15 September 2010] [13:47:35] <guido_g> oh...
[Wednesday 15 September 2010] [13:50:32] <guido_g> nice one
[Wednesday 15 September 2010] [13:50:54] <guido_g> but can you do it in uml?
[Wednesday 15 September 2010] [13:51:01] <guido_g> ducks for cover
[Wednesday 15 September 2010] [13:52:53] Quit pieterh has left this server (Ping timeout: 276 seconds).
[Wednesday 15 September 2010] [13:53:25] <guido_g> more serious thing: smaller version that fits on one page so it can be printed easily
[Wednesday 15 September 2010] [14:04:28] <keffo> sustrik, What scenario could trigger a deadlock? (in single threaded proc.)
[Wednesday 15 September 2010] [14:59:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [15:09:33] <Samy> sustrik, ping?
[Wednesday 15 September 2010] [15:27:31] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 15 September 2010] [15:27:41] <ModusPwnens> Does zeromq have a lot of start up overhead of initialization?
[Wednesday 15 September 2010] [15:27:46] <ModusPwnens> or initialization*
[Wednesday 15 September 2010] [15:36:47] <ModusPwnens> the only reason I ask is because i keep getting strange results when I benchmark where the throughput is low intially, but then gradually increases
[Wednesday 15 September 2010] [15:47:34] Quit g5q has left this server (Quit: Bye~).
[Wednesday 15 September 2010] [15:51:53] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 15 September 2010] [15:54:03] Quit keffo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [16:19:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 15 September 2010] [16:36:23] Quit zirpu has left this server (Quit: kernel upgrade.).
[Wednesday 15 September 2010] [16:44:47] Quit ekidd has left this server (Quit: ekidd).
[Wednesday 15 September 2010] [18:45:35] Quit sbahra has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [19:27:16] Quit danielqo has left this server (Quit: danielqo).
[Wednesday 15 September 2010] [19:56:19] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [20:24:19] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [21:17:56] Quit carrus85 has left this server (Quit: Leaving).
[Wednesday 15 September 2010] [21:25:08] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 15 September 2010] [21:33:25] Quit andrewvc has left this server (Quit: andrewvc).
[Wednesday 15 September 2010] [21:39:21] Quit danielqo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [21:42:30] Quit danielqo has left this server (Remote host closed the connection).
[Wednesday 15 September 2010] [21:48:19] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Wednesday 15 September 2010] [21:52:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 15 September 2010] [22:17:24] <lestrrat> is there some sort of debug flag to enable tracing or some such for zeromq?
[Wednesday 15 September 2010] [22:17:46] <lestrrat> I’m having problems, and want to know what’s going on in the zeromq side...
[Wednesday 15 September 2010] [22:56:14] <dermoth> lestrrat, maybe strace would help, but it might be a bit low-level depending on what you need to figure out
[Wednesday 15 September 2010] [22:57:28] <dermoth> lestrrat, fwiw if you’re using short-lived processes then some queues have no way to guarantee all messages have been delivered before shutdown; when you call zmq_send data is only bufered in a local queue to be sent to the remote end
[Wednesday 15 September 2010] [23:03:52] <lestrrat> thanks, gonna have to RFE a TRACE variable or something ;)
[Thursday 16 September 2010] [00:09:44] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [01:06:23] Quit carrus85 has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [01:28:04] Quit guido_g has left this server (Quit: Boom...).
[Thursday 16 September 2010] [01:35:08] Quit carrus85 has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [01:36:14] Quit carrus85 has left this server (Client Quit).
[Thursday 16 September 2010] [01:52:22] Quit keffo has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [01:58:44] Quit
AndrewBC_ has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [02:03:48] Quit AndrewBC has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [03:02:56] Quit kleppari has left this server (Read error: Operation timed out).
[Thursday 16 September 2010] [03:12:21] <dermoth> I was trying out the php-zmq module today, and it appears some tests die with an assertion failed with –enable-pgm in the zeromq library
[Thursday 16 September 2010] [03:12:43] <dermoth> not that I need it though...
[Thursday 16 September 2010] [03:14:05] <dermoth> the error is: Assertion failed: !pgm_supported () (zmq.cpp:240)
[Thursday 16 September 2010] [03:17:20] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [03:33:05] Quit kenkeiter has left this server (Quit: kenkeiter).
[Thursday 16 September 2010] [03:34:16] Quit __moore__ has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [03:47:30] <sustrik> dermoth: don’t ope 2 contexts
[Thursday 16 September 2010] [03:47:33] <sustrik> open
[Thursday 16 September 2010] [04:03:48] Nick Nickname already in use. Trying
travlr_.
[Thursday 16 September 2010] [04:03:49] Notice -NickServ- travlr_ is not a registered nickname.
[Thursday 16 September 2010] [04:03:50] CTCP Received Version request from frigg.
[Thursday 16 September 2010] [04:03:51] Topic The channel topic is “Welcome!”.
[Thursday 16 September 2010] [04:03:53] Quit travlr has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [04:03:57] Mode Channel modes: no messages from outside, topic protection
[Thursday 16 September 2010] [04:03:57] Created This channel was created on 2010-02-05 10:44.
[Thursday 16 September 2010] [04:09:59] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Thursday 16 September 2010] [04:22:28] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 16 September 2010] [04:23:19] Quit xla has left this server (Client Quit).
[Thursday 16 September 2010] [04:28:49] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 16 September 2010] [04:32:25] Nick Nickname already in use. Trying
travlr__.
[Thursday 16 September 2010] [04:32:26] Notice -NickServ- travlr__ is not a registered nickname.
[Thursday 16 September 2010] [04:32:27] CTCP Received Version request from frigg.
[Thursday 16 September 2010] [04:32:28] Topic The channel topic is “Welcome!”.
[Thursday 16 September 2010] [04:32:34] Mode Channel modes: no messages from outside, topic protection
[Thursday 16 September 2010] [04:32:34] Created This channel was created on 2010-02-05 10:44.
[Thursday 16 September 2010] [04:32:46] Quit
travlr_ has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [04:58:19] Quit xrfang has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [05:26:25] <lestrrat> cool, it wasn’t just me who got bit with multiple contexts + pgm ;)
[Thursday 16 September 2010] [06:05:00] <sustrik> lestrrat: steven mc coy have already started migrating 0mq to new version of openpgm that should have the problem solved
[Thursday 16 September 2010] [06:05:19] <sustrik> but it’s work in progress, no guarantees
[Thursday 16 September 2010] [07:18:29] Quit jsimmons has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [08:03:25] Quit AndrewBC has left this server (Ping timeout: 252 seconds).
[Thursday 16 September 2010] [09:05:38] <psino> in zmq 2.1, zmq_close wont discard that is queued for sending – is anything similar going to be implemented for data that has arrived, and are pending recv-calls?
[Thursday 16 September 2010] [09:21:00] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [09:30:00] Quit guido_g has left this server (Quit: Boom...).
[Thursday 16 September 2010] [09:44:05] <joyenlee> Hi!
[Thursday 16 September 2010] [09:44:52] Quit joyenlee has left this server (Quit: Page closed).
[Thursday 16 September 2010] [09:49:06] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 16 September 2010] [10:00:22] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [10:02:55] Quit iFire has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [10:30:31] Quit mato has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [10:35:09] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [10:37:42] Quit grahamalot has left this server (Read error: Operation timed out).
[Thursday 16 September 2010] [11:33:31] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [11:35:38] Quit psino has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [11:56:30] Quit psino has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [12:45:29] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [13:48:38] Quit AndrewBC has left this server (Ping timeout: 264 seconds).
[Thursday 16 September 2010] [14:02:55] Quit xla has left this server (Quit: leaving).
[Thursday 16 September 2010] [14:14:38] Quit psino has left this server (Quit: psino).
[Thursday 16 September 2010] [14:17:11] Quit keffo has left this server (Read error: Connection reset by peer).
[Thursday 16 September 2010] [14:17:17] Quit ModusPwnens has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [14:32:59] Quit AndrewBC has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [14:45:08] Quit AndrewBC has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [15:07:01] Join yrashk has joined this channel (
~yrashk@70.28.245.3).
[Thursday 16 September 2010] [15:09:10] Quit fredix has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [15:09:22] Quit omarkj has left this server (Quit: omarkj).
[Thursday 16 September 2010] [15:14:22] Join fredix has joined this channel (
~fredix@2a01:e35:2f38:d790:218:2dff:fe00:173).
[Thursday 16 September 2010] [15:56:08] Quit yrashk has left this server (Ping timeout: 276 seconds).
[Thursday 16 September 2010] [17:15:50] <dermoth> sustrik, i didn’t write the test... but why would it fail? What is multiple applications use different context on the same server? do you think the contect should be abstracted in the PHP module?
[Thursday 16 September 2010] [17:43:05] Quit g5q has left this server (Quit: Bye~).
[Thursday 16 September 2010] [18:12:59] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Thursday 16 September 2010] [18:13:23] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [18:36:54] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [18:42:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [18:46:48] Quit keffo has left this server (Ping timeout: 245 seconds).
[Thursday 16 September 2010] [19:13:13] Quit andrewvc has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [19:21:52] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [20:11:41] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 16 September 2010] [20:15:04] Quit jhawk28 has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [20:45:07] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Thursday 16 September 2010] [20:46:51] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 16 September 2010] [20:57:43] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [21:01:31] Quit plq has left this server (Ping timeout: 240 seconds).
[Thursday 16 September 2010] [21:23:48] Quit bgranger has left this server (Quit: bgranger).
[Thursday 16 September 2010] [21:24:04] Quit jhawk28 has left this server (Remote host closed the connection).
[Thursday 16 September 2010] [21:31:14] Quit erickt has left this server (Ping timeout: 272 seconds).
[Thursday 16 September 2010] [21:41:14] <
mattp_> Hi. I’m trying to build using Cygwin. I’m getting me an error message telling me to install e2fsprog even though I’ve already got it.
[Thursday 16 September 2010] [21:41:18] <
mattp_> Any suggestions?
[Thursday 16 September 2010] [21:42:22] <
mattp_> (not sure how that “me” got into my third sentence. :) )
[Thursday 16 September 2010] [21:49:23] <
mattp_> crickets...
[Thursday 16 September 2010] [21:50:38] Quit
mattp_ has left this server (Quit: Using leafChat 2).
[Thursday 16 September 2010] [21:51:44] Quit
mattp_ has left this server (Client Quit).
[Thursday 16 September 2010] [21:52:32] <mpassell> thought I’d try through a different client. Anyone out there?
[Thursday 16 September 2010] [22:01:10] Part mpassell has left this channel.
[Thursday 16 September 2010] [22:12:57] Quit ekidd has left this server (Quit: ekidd).
[Thursday 16 September 2010] [23:28:16] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Thursday 16 September 2010] [23:51:02] <chowmeined> can one multiplex different message patterns on a single socket?
[Friday 17 September 2010] [00:18:42] <andrewvc> cremes, by chance are you around?
[Friday 17 September 2010] [00:19:25] Part snerd has left this channel.
[Friday 17 September 2010] [00:33:45] <dermoth> iirc there used to be device config example, looks like they’re gone. why?
[Friday 17 September 2010] [00:33:56] <dermoth> king of easier than reading the code :)
[Friday 17 September 2010] [00:38:41] <guido_g> maybe thats what you want
[Friday 17 September 2010] [00:40:27] <dermoth> guido_g, thanks, looks nice. Actually for now i’m just using the stock devices that come with zeromq. The config quite basic, in xml. I was justt wondering where they gone
[Friday 17 September 2010] [00:40:36] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [00:41:11] <guido_g> afair there going to be moved out of the core distribution
[Friday 17 September 2010] [00:42:44] <dermoth> ok... well I kind of like them for the most basic usage... I could replace the config with command-line arguments though, imho it would be much easier
[Friday 17 September 2010] [00:43:05] <guido_g> dermoth: btw, please send your patch to the ml with a statement that it is licensed under mit/x11, othwerwise it can’t be accepted
[Friday 17 September 2010] [00:43:33] <dermoth> really?
[Friday 17 September 2010] [00:44:25] <guido_g> it’s the procedure
[Friday 17 September 2010] [00:44:50] <guido_g> has been discussed here and on the ml
[Friday 17 September 2010] [00:46:13] <guido_g> dermoth: easiest way is to use format-patch and append that to the mal w/ the license statement
[Friday 17 September 2010] [00:46:15] <dermoth> then how about I sen you this?
[Friday 17 September 2010] [00:46:19] <dermoth> perl -pi -e ‘s/ZMQ_DOWNSTREAM/ZMQ_PUSH/; s/ZMQ_UPSTREAM/ZMQ_PULL/;’ devices/zmq_streamer/zmq_streamer.cpp
[Friday 17 September 2010] [00:46:20] <guido_g> not me
[Friday 17 September 2010] [00:46:36] <guido_g> dermoth: see git format-patch
[Friday 17 September 2010] [00:46:50] <dermoth> git commit -m’zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL’
[Friday 17 September 2010] [00:47:12] <dermoth> I use git-format-patch... but I’m not even subscrived to the ml
[Friday 17 September 2010] [00:47:29] <guido_g> too bad
[Friday 17 September 2010] [00:47:35] <dermoth> the two commands abot will get you the same result less the credit
[Friday 17 September 2010] [00:48:02] <guido_g> what credit?
[Friday 17 September 2010] [00:48:11] <dermoth> the author in commit message
[Friday 17 September 2010] [00:48:33] <guido_g> ah
[Friday 17 September 2010] [00:48:55] <dermoth> unless you’re authorised to do: git commit -m’zmq_streamer: Rename DOWNSTREAM/UPSTREAM to PUSH/PULL’ ‘–author=Thomas Guyot-Sionnest <
thomas@blinkx.com>’
[Friday 17 September 2010] [00:48:56] <guido_g> i’m more concerned about the license statement, w/o it the patch can’t go in
[Friday 17 September 2010] [00:49:40] <dermoth> it’s not a patch it a command that patsh, there shouldn’t be any legal issue. if you run the command you have the copyright for the resulting code ;)
[Friday 17 September 2010] [00:50:13] <guido_g> i think that is clear
[Friday 17 September 2010] [00:50:50] <guido_g> i just stated what’s needed to get a patch in, because a pull-req is a patch
[Friday 17 September 2010] [00:54:01] Quit kenkeiter has left this server (Quit: kenkeiter).
[Friday 17 September 2010] [00:54:07] <dermoth> I understand the legal basis, but I will likely not send a patch that takes me more time to “send” thna write... Imagine trying that with the Linux Kernel!
[Friday 17 September 2010] [00:54:48] <guido_g> as i said, the device apps will be moved out of the core, afaik
[Friday 17 September 2010] [00:55:01] <guido_g> so...
[Friday 17 September 2010] [00:55:12] <dermoth> so I guess there won’t be any trouble getting that in, whatever method you use... on that i’ll get back to my testing :)
[Friday 17 September 2010] [00:55:25] <guido_g> dermoth: wrong
[Friday 17 September 2010] [00:55:35] <guido_g> the license issue is a problem atm
[Friday 17 September 2010] [00:55:41] <guido_g> ask pieter about it
[Friday 17 September 2010] [00:56:17] <guido_g> and go read the ml (via gmane w/o registration)
[Friday 17 September 2010] [00:56:51] <dermoth> what I meant is that it probably take you less time to rename it yourself than using my patch (and I use gmane...)
[Friday 17 September 2010] [00:57:57] <dermoth> i wouldn’t mind taking the time to post on the ml for a real change... and can’T I put it in the commit message too?
[Friday 17 September 2010] [00:58:14] <guido_g> cool, then you can easily find the threads about patches and legal issues
[Friday 17 September 2010] [00:58:43] <guido_g> if it were in the commit message, even better
[Friday 17 September 2010] [00:58:44] <dermoth> except that i’d rather catch some much needed sleep
[Friday 17 September 2010] [00:58:48] <dermoth> ok
[Friday 17 September 2010] [01:04:50] <dermoth> here you go... next time i’ll know :)
[Friday 17 September 2010] [01:12:46] <guido_g> hey, i didn’t invent the laws and such, just repeating the handbook
[Friday 17 September 2010] [01:13:07] <dermoth> Any PECL developers here? IMHO the PHP api should have a cb that is called on persistent socket creation, so we could create the socket and connect in one shot
[Friday 17 September 2010] [01:13:24] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [01:33:13] Quit carrus85 has left this server (Quit: Leaving).
[Friday 17 September 2010] [01:33:57] Join kenkeiter has joined this channel (
~kenkeiter@2002:4c73:685:1234:223:6cff:fe98:be3b).
[Friday 17 September 2010] [01:36:54] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [01:37:00] Quit kenkeiter has left this server (Client Quit).
[Friday 17 September 2010] [01:40:32] <sleeperbot> this driver takes up a ton of CPU
[Friday 17 September 2010] [01:42:32] <sleeperbot> the way that it is implemented =/
[Friday 17 September 2010] [01:48:29] <yrashk> sleeperbot afair there’s none
[Friday 17 September 2010] [01:48:41] <yrashk> but supposedly I will need one soon-ish
[Friday 17 September 2010] [01:48:54] <yrashk> so might invest some time in getting it done properly
[Friday 17 September 2010] [01:51:51] <sleeperbot> i see, thanks for the info
[Friday 17 September 2010] [01:52:55] <yrashk> yw
[Friday 17 September 2010] [01:54:32] Quit keffo has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [02:20:26] Quit sleeperbot has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [02:44:10] Quit jsimmons has left this server (Client Quit).
[Friday 17 September 2010] [02:52:57] Quit xrfang has left this server (Remote host closed the connection).
[Friday 17 September 2010] [03:02:27] <sustrik> psino: how would it work for recv calls?
[Friday 17 September 2010] [03:03:38] <sustrik> dermoth: it’s a problem with OpenPGM, which will be solved soon; what i was saying is that in most cases you have just one context so you never encounter the problem
[Friday 17 September 2010] [03:04:38] <sustrik> chowmeined: no, you can’t
[Friday 17 September 2010] [03:04:59] Quit kleppari has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [03:06:05] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Friday 17 September 2010] [03:09:17] Quit guido_g has left this server (Quit: Boom...).
[Friday 17 September 2010] [03:21:08] Quit fredix has left this server (Read error: Operation timed out).
*** on Fri Sep 17 04:13:00 2010
[Friday 17 September 2010] [04:13:00] Topic The channel topic is “Welcome!”.
[Friday 17 September 2010] [04:13:04] Mode Channel modes: no messages from outside, topic protection
[Friday 17 September 2010] [04:13:04] Created This channel was created on 2010-02-05 10:44.
[Friday 17 September 2010] [04:49:41] <ivan-korobkov> Hi, I use zmq 2.0.9 + epgm + the latest python bindings. Sometimes it crashes with Assertion failed:
*tmpbuf > 0 (zmq_decoder.cpp:60). Does anybody know why it happens? Is it a bug and I should create an issue?
[Friday 17 September 2010] [05:05:08] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 17 September 2010] [05:08:09] <guido_g> try to isolate it in a small test case, so others can verify it
[Friday 17 September 2010] [05:12:03] <ivan-korobkov> OK, I will try, but it happens rather sporadically.
[Friday 17 September 2010] [05:13:15] <guido_g> will be no problem, if we have a program that can run
[Friday 17 September 2010] [05:17:15] Quit xrfang has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [05:31:16] <psino> sustrik: In the case of a pull socket, I was thinking it could disconnect from the push socket and still be able to recv() on the messages the push socket has sent to it
[Friday 17 September 2010] [05:40:43] <sustrik> ivan-korobkov: create an issue
[Friday 17 September 2010] [05:41:15] <sustrik> psino: any idea of how the API should look like?
[Friday 17 September 2010] [05:46:23] <psino> sustrik: from the end users point of view, much like now, except that after calling socket.close(), a poller wouldn’t put the socket in exceptional state before the messages that has been sent to it from the other side has been recv()’ed
[Friday 17 September 2010] [05:49:07] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [05:56:48] Quit mato has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:10:01] Quit ivan-korobkov has left this server (Quit: Leaving).
[Friday 17 September 2010] [06:15:51] Quit guido_g has left this server (Quit: Boom...).
[Friday 17 September 2010] [06:16:50] Quit iFire has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:28:18] Quit
AndrewBC_ has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:28:19] Quit plq has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:32:59] <CIA-20> zeromq2: 03Martin Sustrik 07master * r85aa25e 10/ src/req.cpp : bug in REQ+multipart fixed -
http://bit.ly/bzWD3V[Friday 17 September 2010] [06:39:48] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [06:46:52] Quit omarkj has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [06:46:53] Nick
omarkj_ is now known as omarkj.
[Friday 17 September 2010] [06:47:41] Quit pieterh has left this server (Quit: Leaving).
[Friday 17 September 2010] [06:50:32] Quit mato has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [06:58:32] <CIA-20> zeromq2: 03Martin Sustrik 07master * rc7b9ba3 10/ (src/command.hpp src/session.cpp): reconnection process fixed when failure occurs during init phase -
http://bit.ly/cIBUcl[Friday 17 September 2010] [06:58:58] Quit
AndrewBC_ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [08:10:38] Quit GeekGod has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [08:50:47] Quit sbahra has left this server (Remote host closed the connection).
[Friday 17 September 2010] [10:05:01] Join plq has joined this channel (
~plq@78.189.30.218).
[Friday 17 September 2010] [10:09:28] Quit bgranger has left this server (Quit: bgranger).
[Friday 17 September 2010] [10:27:57] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Friday 17 September 2010] [10:32:15] Quit plq has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [12:28:09] Quit cremes has left this server (Quit: cremes).
[Friday 17 September 2010] [12:28:48] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Friday 17 September 2010] [12:34:43] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [12:36:18] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [13:29:54] Quit yrashk has left this server (Ping timeout: 246 seconds).
[Friday 17 September 2010] [13:39:04] Quit omarkj has left this server (Quit: omarkj).
[Friday 17 September 2010] [14:48:58] Quit ModusPwnens has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [14:51:00] Quit __moore__ has left this server (Remote host closed the connection).
[Friday 17 September 2010] [14:55:33] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [15:00:22] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [15:05:32] Quit kelvink has left this server (Quit: Leaving.).
[Friday 17 September 2010] [16:02:02] <bgranger> sustrik: question about msg ref counting...
[Friday 17 September 2010] [16:11:22] <keffo> I think they’re all off work :)
[Friday 17 September 2010] [16:11:32] <keffo> starting to think I’m the only one around anymore :)
[Friday 17 September 2010] [16:12:29] <keffo> lol, last time anyone even spoke in here was about 11h ago :)
[Friday 17 September 2010] [16:12:39] <andrewvc> and you just broke the record
[Friday 17 September 2010] [16:12:55] <keffo> no, bgranger did 10 mins ago :)
[Friday 17 September 2010] [16:14:09] <andrewvc> oh damn, hehe, that’s what I get for sleeping my laptop, missing all the action
[Friday 17 September 2010] [16:16:43] <keffo> :)
[Friday 17 September 2010] [16:17:16] <keffo> Does anyone know best-practices for transferring large amounts of data? Simply to split into multipart?
[Friday 17 September 2010] [16:17:19] Quit yrashk has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [16:17:46] <andrewvc> I remember there were benchmarks on the zeromq site with various message sizes, ranging up to something huge
[Friday 17 September 2010] [16:18:22] <keffo> right, but simply sending one huge message just sounds like mem hog to me
[Friday 17 September 2010] [16:29:58] Quit xla has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [16:32:34] <sustrik> i am back
[Friday 17 September 2010] [16:32:49] <sustrik> bgranger: what’s the question?
[Friday 17 September 2010] [16:34:24] <sustrik> keffo: so is multipart message, you have to break it into sequence of messages
[Friday 17 September 2010] [16:34:42] <sustrik> then you have to define your own transfer protocol etc.
[Friday 17 September 2010] [16:34:45] <sustrik> it’s not easy
[Friday 17 September 2010] [16:34:51] <keffo> mm
[Friday 17 September 2010] [16:35:01] <sustrik> what do you want to do?
[Friday 17 September 2010] [16:35:31] <keffo> To be honest I’m leaning towards an accompanying torrent lib for larger transfers
[Friday 17 September 2010] [16:35:46] <keffo> I solved both my deadlock and garbled data btw..
[Friday 17 September 2010] [16:35:59] <keffo> as expected it was all me :)
[Friday 17 September 2010] [16:36:03] <keffo> CRT mismatch actually
[Friday 17 September 2010] [16:36:06] <sustrik> :)
[Friday 17 September 2010] [16:36:34] <sustrik> yeah, makes sense, you would have to implement torrent kind of behaiour yourself on top of 0mq
[Friday 17 September 2010] [16:36:55] <bgranger> sustrik: we have a very subtle question about ref counting of msgs...
[Friday 17 September 2010] [16:37:16] <keffo> sustrik, I was thinking more along the lines of using the transmission codebase, or libtorrent..
[Friday 17 September 2010] [16:37:57] <sustrik> bgranger: yes?
[Friday 17 September 2010] [16:38:00] Join yrashk has joined this channel (
~yrashk@70.28.245.5).
[Friday 17 September 2010] [16:38:10] <bgranger> Hi
[Friday 17 September 2010] [16:38:11] <sustrik> keffo: sure
[Friday 17 September 2010] [16:38:14] <sustrik> hi
[Friday 17 September 2010] [16:38:22] <bgranger> We are trying to understand the refcounting
[Friday 17 September 2010] [16:38:40] <bgranger> We see that zmq_msg_close is the place where the ffn is called if the ref count is 0
[Friday 17 September 2010] [16:39:10] <bgranger> What exactly does zmq_send do with the ref count. What is zmq_close is called before a msg is actually send?
[Friday 17 September 2010] [16:39:26] <bgranger> Does send somehow end up calling zmq_msg_close?
[Friday 17 September 2010] [16:42:42] <sustrik> yes, send takes ownership of your message
[Friday 17 September 2010] [16:43:03] <bgranger> OK, we just saw that at the end of xsend there is
[Friday 17 September 2010] [16:43:07] <sustrik> your message object will be empty after return from the send call
[Friday 17 September 2010] [16:43:10] <bgranger> int rc = zmq_msg_init (
msg_);
[Friday 17 September 2010] [16:43:30] <sustrik> yes, that initialises your message to empty
[Friday 17 September 2010] [16:43:31] <bgranger> So if you then call zmq_msg_close you are not closing the original msg, but a new empty one.
[Friday 17 September 2010] [16:43:38] <bgranger> Great!
[Friday 17 September 2010] [16:43:39] <sustrik> exactly
[Friday 17 September 2010] [16:43:59] <bgranger> We are getting python to know when all copies of a message are done being sent.
[Friday 17 September 2010] [16:44:29] <sustrik> intercepting the free function?
[Friday 17 September 2010] [16:44:41] <bgranger> There is a subtle issue with non-copy sends. The user has to be able to know when zmq is done with it and they can edit the buffer again.
[Friday 17 September 2010] [16:44:46] <bgranger> Yes, we use the ffn.
[Friday 17 September 2010] [16:45:13] <bgranger> Otherwise the user might accidentally start the edit the buffer they sent before zmq has send it...
[Friday 17 September 2010] [16:45:29] <bgranger> non-copy sends are super subtle...
[Friday 17 September 2010] [16:45:34] <sustrik> so you are basically writing your own allocator
[Friday 17 September 2010] [16:46:21] <bgranger> We use Pythons garbage collection mechanism along with our own ref counts that allow us to track the underlying zmq ref counts.
[Friday 17 September 2010] [16:46:45] <sustrik> what’s the goal?
[Friday 17 September 2010] [16:47:12] <bgranger> In Python you might have a mutable array of numbers that you want to send using non-copy.
[Friday 17 September 2010] [16:47:27] <bgranger> You send it, and maybe then you want to modify it.
[Friday 17 September 2010] [16:47:37] <sustrik> i see
[Friday 17 September 2010] [16:47:40] <bgranger> But if you modify it before zmq sends it, the wrong thing gets sent
[Friday 17 September 2010] [16:47:54] <bgranger> This is common in Python with numerical arrays (matrices, etc.)
[Friday 17 September 2010] [16:48:23] <bgranger> For large static data it is not a problem, but in numerical/scientific computing, most data structures are mutable...
[Friday 17 September 2010] [16:49:08] <bgranger> sustrik: I think we understand the refcounting better now though. Thanks!
[Friday 17 September 2010] [16:50:55] <sustrik> np
[Friday 17 September 2010] [16:51:04] <sustrik> a question though:
[Friday 17 September 2010] [16:51:33] <sustrik> doesn’t it turn the messaging into sync model?
[Friday 17 September 2010] [16:51:43] <sustrik> one-message-a-time?
[Friday 17 September 2010] [17:04:34] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [17:08:59] <bgranger> sustrik: back from lunch...
[Friday 17 September 2010] [17:09:15] <bgranger> no the messaging is still fully async
[Friday 17 September 2010] [17:09:21] <minrk> sustrik: I’m not sure what your question means. All we did was add an object that tracks whether a message no longer has references in zmq
[Friday 17 September 2010] [17:09:48] <bgranger> You can then query that object to learn if the message has fully been sent.
[Friday 17 September 2010] [17:10:03] <bgranger> send does not block any more than it used to
[Friday 17 September 2010] [17:10:29] <minrk> or if there are any message objects pointing to it at all
[Friday 17 September 2010] [17:11:15] <bgranger> minrk: it would be nice if we could have a method of MessageTracker that would block until the msg is not pending though.
[Friday 17 September 2010] [17:12:56] <minrk> that’s easy
[Friday 17 September 2010] [17:13:34] Quit keffo has left this server (Ping timeout: 255 seconds).
[Friday 17 September 2010] [17:13:42] <bgranger> minrk: I am ready when you are....just give me a call on skype
[Friday 17 September 2010] [17:13:50] <minrk> okay
[Friday 17 September 2010] [17:14:08] <minrk> give me a few minutes
[Friday 17 September 2010] [17:15:27] <bgranger> no problem...
[Friday 17 September 2010] [17:20:09] <minrk> okay
[Friday 17 September 2010] [17:22:24] <mrossi> anyone know who is on the sending side of the signaler instance inside of the app_thread class?
[Friday 17 September 2010] [17:24:07] <mrossi> have an app that occasionally gets hung at line 88 of app_thread.cpp doing a signaler.recv()
[Friday 17 September 2010] [17:38:05] <dbudworth> question from a newbie zeromq user. I’d like to implement a socket server type system where the server is stateful for each client connection. Is there a zeromq socket type for this? it appears as though the server doesn’t have any context about which client it’s sending to at any one time
[Friday 17 September 2010] [17:46:16] Quit mrossi has left this server (Ping timeout: 252 seconds).
[Friday 17 September 2010] [18:09:29] Quit gavinstark has left this server (Remote host closed the connection).
[Friday 17 September 2010] [18:17:04] Quit
miguelito__ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [18:18:02] Quit skaar has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [18:34:58] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [18:35:02] Part minrk has left this channel.
[Friday 17 September 2010] [18:36:53] Quit dbudworth has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [18:51:56] <ModusPwnens> Hmm, can anyone take a look at some results from benchmarking tests?
[Friday 17 September 2010] [18:52:16] <ModusPwnens> I am seeing some strange results, but maybe they are normal
[Friday 17 September 2010] [19:25:30] Quit bgranger has left this server (Read error: Operation timed out).
[Friday 17 September 2010] [19:40:28] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Friday 17 September 2010] [19:48:48] Quit rossij has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [20:05:09] Quit yrashk has left this server (Ping timeout: 240 seconds).
[Friday 17 September 2010] [20:46:37] Quit jhawk28 has left this server (Remote host closed the connection).
[Friday 17 September 2010] [20:50:24] Quit andrewvc has left this server (Quit: andrewvc).
[Friday 17 September 2010] [20:52:20] Quit bennymack has left this server (Write error: Broken pipe).
[Friday 17 September 2010] [21:15:43] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Friday 17 September 2010] [21:27:38] Quit ModusPwnens has left this server (Ping timeout: 272 seconds).
[Friday 17 September 2010] [21:50:26] Quit yrashk has left this server (Ping timeout: 264 seconds).
[Friday 17 September 2010] [22:10:08] Quit chowmeined has left this server (Read error: Connection reset by peer).
[Friday 17 September 2010] [22:14:32] Quit chowmeined has left this server (Client Quit).
[Friday 17 September 2010] [23:01:11] Quit omarkj has left this server (Quit: omarkj).
[Friday 17 September 2010] [23:08:31] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:25:41] Quit
aleator_ has left this server (Ping timeout: 276 seconds).
[Friday 17 September 2010] [23:27:02] Quit bgranger has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:27:02] Quit bennymac1 has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:27:02] Quit Tasser has left this server (Ping timeout: 265 seconds).
[Friday 17 September 2010] [23:38:47] <benford> good morning folks. can anyone answer a question about non blocking recieve and send?
[Friday 17 September 2010] [23:45:02] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [00:02:58] Quit beppu has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [00:03:18] Quit skaar has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [00:19:11] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Saturday 18 September 2010] [01:42:31] Quit yrashk has left this server (Ping timeout: 255 seconds).
[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r0c215fa 10/ (src/fq.cpp src/lb.cpp src/socket_base.cpp): potential duplicate termination of pipes removed -
http://bit.ly/9s1gJt[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * rfb6ce53 10/ src/own.cpp : more details on launch_sibling in comments -
http://bit.ly/bQt0hE[Saturday 18 September 2010] [02:16:47] <CIA-20> zeromq2: 03Martin Sustrik 07master * r4c6d07d 10/ (15 files): single term ack counting mechanism for every socket (no separate mechanisms for fq_t and lb_t) -
http://bit.ly/cS50VY[Saturday 18 September 2010] [02:17:24] <sustrik> benford: shoot
[Saturday 18 September 2010] [02:17:33] Join yrashk has joined this channel (
~yrashk@70.28.245.4).
[Saturday 18 September 2010] [02:18:44] <benford> hi, i think it was user error :-) i’m trying to wrap the concurrent eventlet library around socket recv/send and was having some tryouble with polling
[Saturday 18 September 2010] [02:20:03] <sustrik> ok
[Saturday 18 September 2010] [02:29:55] <benford> actually - i guess i’ve got another question: if i’m doing a zmq_poll what might throw errno 14 ‘Bad address’
[Saturday 18 September 2010] [02:30:36] <benford> i’m probably calling this with an int representing an eventlet socket so i’m thinking that the flags are causing some sort of error?
[Saturday 18 September 2010] [02:30:57] <benford> (this is in pyzmq btw)
[Saturday 18 September 2010] [02:34:21] Quit bgranger has left this server (Quit: bgranger).
[Saturday 18 September 2010] [02:34:37] <benford> when using zmq_poll do i still have to stick with the POLLIN and POLLOUT flags or would i need to replace them with the equvalent select flags?
[Saturday 18 September 2010] [02:37:34] <benford> doh - figured it - was using a different context in the same thread
[Saturday 18 September 2010] [02:45:29] Quit mikko has left this server (Remote host closed the connection).
[Saturday 18 September 2010] [03:02:32] Quit kleppari has left this server (Read error: Operation timed out).
[Saturday 18 September 2010] [03:11:13] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Saturday 18 September 2010] [05:05:21] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [05:39:22] Quit kelvink has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [05:51:08] Quit yrashk has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [05:55:50] Join yrashk has joined this channel (
~yrashk@70.28.245.1).
[Saturday 18 September 2010] [06:05:08] Quit benford has left this server (Quit: benford).
[Saturday 18 September 2010] [06:08:04] Quit xla has left this server (Ping timeout: 272 seconds).
[Saturday 18 September 2010] [06:24:34] Quit mitkok has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [06:41:30] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [06:42:54] Quit yrashk has left this server (Ping timeout: 272 seconds).
[Saturday 18 September 2010] [06:43:58] Quit
AndrewBC_ has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [07:40:04] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [08:25:40] Quit xla has left this server (Quit: leaving).
[Saturday 18 September 2010] [09:06:58] Quit plq has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [09:20:20] Quit keffo has left this server (Quit: Leaving).
[Saturday 18 September 2010] [09:21:27] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [10:11:13] Quit benford has left this server (Client Quit).
[Saturday 18 September 2010] [10:14:33] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 18 September 2010] [10:17:41] Quit psino has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [10:45:41] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [11:25:25] Part mr19 has left this channel.
[Saturday 18 September 2010] [11:29:48] Quit mitkok has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [12:08:35] Quit plq has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [12:12:08] Quit bgranger has left this server (Quit: bgranger).
[Saturday 18 September 2010] [12:20:41] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [12:37:13] Quit rossij has left this server (Ping timeout: 245 seconds).
[Saturday 18 September 2010] [13:08:53] Quit xla has left this server (Ping timeout: 245 seconds).
[Saturday 18 September 2010] [13:15:58] Quit benford has left this server (Quit: benford).
[Saturday 18 September 2010] [13:52:50] Quit benford has left this server (Client Quit).
[Saturday 18 September 2010] [14:16:48] Quit mitkok has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [15:24:28] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [15:28:43] Quit plq has left this server (Ping timeout: 240 seconds).
[Saturday 18 September 2010] [15:30:08] Quit ModusPwnens has left this server (Ping timeout: 276 seconds).
[Saturday 18 September 2010] [15:32:21] Join plq has joined this channel (
~plq@88.249.173.198).
[Saturday 18 September 2010] [16:01:51] <mr19> docs say to call zmq_init() from the “start of your main line of code”, my app starts a new thread that calls zmq_init() (via the zmq::context_t constructor) at thread startup. can easily be switched but curious as to whether this can cause problems. quick look through the code didn’t raise any flags to me.
[Saturday 18 September 2010] [16:06:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Saturday 18 September 2010] [16:34:20] <sustrik> mr19: to be able to communicate in inter-thread manner you have t share the context
[Saturday 18 September 2010] [16:34:42] <sustrik> i mean inproc:// transport
[Saturday 18 September 2010] [17:15:25] Quit sbahra has left this server (Ping timeout: 252 seconds).
[Saturday 18 September 2010] [17:27:14] <mr19> ok, i’m just using tcp and only from within that thread, just trying to simplify my code – i’m going to move the ctx instantiation to the main thread but was just curious if there are any other reasons for requiring the call to zmq_init() in the main thread
[Saturday 18 September 2010] [17:33:01] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [18:23:34] Quit sbahra has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [18:28:01] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [18:33:34] Quit psino has left this server (Client Quit).
[Saturday 18 September 2010] [19:44:12] Quit kelvink has left this server (Quit: Leaving.).
[Saturday 18 September 2010] [19:47:08] Quit psino has left this server (Quit: psino).
[Saturday 18 September 2010] [20:11:50] Quit bbigras has left this server (Read error: Connection reset by peer).
[Saturday 18 September 2010] [21:00:12] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Saturday 18 September 2010] [22:16:16] Part mr19 has left this channel.
[Saturday 18 September 2010] [23:35:34] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Saturday 18 September 2010] [23:57:08] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [00:41:43] <dbudworth> is there a socket type that’s usable for request/streaming response type communications? ie: N clients open connection to a server and the server has a unique conversation with each client. (similar to regular socket accept/listen model)
[Sunday 19 September 2010] [01:09:28] <sustrik> if you want to control each socket, you need a regular socket
[Sunday 19 September 2010] [02:28:26] Quit bgranger has left this server (Quit: bgranger).
[Sunday 19 September 2010] [03:04:46] Quit kleppari has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [03:06:08] <CIA-20> zeromq2: 03Martin Sustrik 07master * r5153b63 10/ (src/session.cpp src/session.hpp): obsolete ‘active’ flag removed from session_t -
http://bit.ly/cRgwVE[Sunday 19 September 2010] [03:06:08] <CIA-20> zeromq2: 03Martin Sustrik 07master * r1f10208 10/ src/session.cpp : termination of pipe via delimiter message could stuck when no data were read from the pipe (because connection wasn’t active) – fixed -
http://bit.ly/dpganD[Sunday 19 September 2010] [03:06:18] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Sunday 19 September 2010] [03:12:19] Quit ikolomiets has left this server (Ping timeout: 252 seconds).
[Sunday 19 September 2010] [04:18:53] Quit mitkok has left this server (Ping timeout: 245 seconds).
[Sunday 19 September 2010] [04:51:01] Quit AndrewBC has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [05:53:38] Quit psino has left this server (Changing host).
[Sunday 19 September 2010] [05:53:41] Quit guido_g has left this server (Excess Flood).
[Sunday 19 September 2010] [06:43:04] Quit psino has left this server (Quit: psino).
[Sunday 19 September 2010] [08:00:05] Quit keffo has left this server (Ping timeout: 255 seconds).
[Sunday 19 September 2010] [09:02:38] Quit plq has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [09:22:03] Quit xla has left this server (Quit: leaving).
[Sunday 19 September 2010] [09:28:25] Quit iFire has left this server (Remote host closed the connection).
[Sunday 19 September 2010] [10:49:40] Quit psino has left this server (Quit: psino).
[Sunday 19 September 2010] [11:14:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 19 September 2010] [12:47:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Sunday 19 September 2010] [12:58:14] Quit keffo has left this server (Ping timeout: 264 seconds).
[Sunday 19 September 2010] [13:12:36] Quit keffo has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [14:02:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Sunday 19 September 2010] [14:27:33] Part sigmonsays has left this channel (“Leaving”).
[Sunday 19 September 2010] [15:15:03] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [15:21:20] Quit keffo has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [15:46:03] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [15:59:58] <CIA-20> zeromq2: 03Martin Sustrik 07master * r471013a 10/ src/zmq_init.cpp : two races fixed in session creation -
http://bit.ly/9UgZMy[Sunday 19 September 2010] [15:59:59] <CIA-20> zeromq2: 03Martin Sustrik 07master * r938689a 10/ (src/ctx.cpp src/zmq_init.cpp): synchronisation problem fixed in ctx_t -
http://bit.ly/bcX8OW[Sunday 19 September 2010] [16:04:03] Part AndrewBC has left this channel (“Leaving”).
[Sunday 19 September 2010] [16:21:47] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 19 September 2010] [16:25:35] Quit mitkok has left this server (Quit: Leaving.).
[Sunday 19 September 2010] [16:27:56] Quit rgl has left this server (Quit: Leaving).
[Sunday 19 September 2010] [16:50:14] <CIA-20> zeromq2: 03Martin Sustrik 07master * rcda3c96 10/ src/ctx.cpp : minor code reorganisation in ctx_t::terminate -
http://bit.ly/96qYvk[Sunday 19 September 2010] [16:50:16] <CIA-20> zeromq2: 03Martin Sustrik 07master * r1d23997 10/ (src/zmq_engine.cpp src/zmq_engine.hpp src/zmq_init.cpp): zmq_init_t destroyed zmq_engine_t before plugging it out from the poller first – fixed -
http://bit.ly/cvBySe[Sunday 19 September 2010] [16:50:38] Quit psino has left this server (Ping timeout: 255 seconds).
[Sunday 19 September 2010] [16:50:38] Nick
psino_ is now known as psino.
[Sunday 19 September 2010] [17:26:49] Quit plq has left this server (Remote host closed the connection).
[Sunday 19 September 2010] [17:58:28] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Sunday 19 September 2010] [18:06:05] Quit mitkok has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [18:06:23] Quit mitkok has left this server (Client Quit).
[Sunday 19 September 2010] [18:07:37] <CIA-20> zeromq2: 03Martin Sustrik 07master * r328c92a 10/ (4 files): problem with engine being attached to session while it’s being terminated fixed -
http://bit.ly/aVsix0[Sunday 19 September 2010] [19:15:32] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [20:01:47] Quit carrus85 has left this server (Quit: Leaving).
[Sunday 19 September 2010] [20:07:30] Quit
miguelito_ has left this server (Ping timeout: 252 seconds).
[Sunday 19 September 2010] [20:39:59] Quit miguelito has left this server (Read error: Connection reset by peer).
[Sunday 19 September 2010] [23:06:30] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Sunday 19 September 2010] [23:20:05] <stoneman> hello
[Sunday 19 September 2010] [23:20:10] Join dirtmcgirt has joined this channel (
~dirtmcgir@2002:62f8:7ff4:0:daa2:5eff:fe98:8f1c).
[Sunday 19 September 2010] [23:20:30] <stoneman> is anyone here?
[Sunday 19 September 2010] [23:21:36] Quit dirtmcgirt has left this server (Client Quit).
[Sunday 19 September 2010] [23:27:17] <dbudworth> Trying to figure out if 0mq is appropriate for my project, simple description would be something like a sticky load balancer. once a client talks to a service, it sticks with that one. for a bi-directional conversation between 2 nodes once an initial node has been selected. ie: my client does round robin on each new conversation but updates to a given conversation have to stick to the beginning server
[Sunday 19 September 2010] [23:27:43] <dbudworth> and hello stoneman, i’m here. but basically not useful if you are looking for help
[Sunday 19 September 2010] [23:35:48] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Sunday 19 September 2010] [23:36:07] Part stoneman has left this channel.
[Monday 20 September 2010] [01:08:51] Quit guido_g has left this server (Quit: Boom...).
[Monday 20 September 2010] [01:33:38] Quit jsimmons has left this server (Ping timeout: 264 seconds).
[Monday 20 September 2010] [01:56:23] Quit keffo has left this server (Ping timeout: 245 seconds).
[Monday 20 September 2010] [02:22:21] Quit benford has left this server (Quit: benford).
[Monday 20 September 2010] [02:54:04] Part benford has left this channel.
[Monday 20 September 2010] [03:04:31] Quit kleppari has left this server (Ping timeout: 246 seconds).
[Monday 20 September 2010] [03:06:28] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Monday 20 September 2010] [04:22:49] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 20 September 2010] [05:10:23] <mato> sustrik: i was just thinking about the zmq_term()/zmq_close() semantics
[Monday 20 September 2010] [05:10:40] <mato> sustrik: and i came up with one small detail
[Monday 20 September 2010] [05:11:03] <mato> sustrik: which may or may not be helpful in the case where zmq_term() would wait indefinitely on exit
[Monday 20 September 2010] [05:11:23] <mato> sustrik: the thing is, syscall close() means “commit”.
[Monday 20 September 2010] [05:12:02] <mato> sustrik: if you don’t close your filedescriptors, and don’t use fsync() (not for sockets obviously), then there is no commitment on the part of the kernel
[Monday 20 September 2010] [05:13:02] <mato> sustrik: hence, as far as sockets with data in-flight at zmq_term() time are concerned, we only need care about those that zmq_close() has actually been called on
[Monday 20 September 2010] [05:13:10] <mato> sustrik: for any other sockets, all bets are off
[Monday 20 September 2010] [05:19:00] <sustrik> mato: sure, but user has to close all the sockets anyway
[Monday 20 September 2010] [05:19:12] <sustrik> otherwise he’ll end up with memory leaks
[Monday 20 September 2010] [05:19:52] <mato> sustrik: i was thinking more of the case of “the application terminating in an abnormal way”
[Monday 20 September 2010] [05:20:03] <sustrik> no guarantees then
[Monday 20 September 2010] [05:20:14] <mato> sure, what i mean is...
[Monday 20 September 2010] [05:20:24] <mato> if an application handles e.g. SIGINT/SIGTERM
[Monday 20 September 2010] [05:20:39] <mato> should it go and call zmq_term() as part of its exit process in such a case?
[Monday 20 September 2010] [05:20:48] <mato> or explicitly not do that, becuase the call may block...?
[Monday 20 September 2010] [05:21:02] <mato> i guess the answer is application dependent
[Monday 20 September 2010] [05:21:21] <sustrik> no idea
[Monday 20 September 2010] [05:21:48] <mato> oh, and another thing
[Monday 20 September 2010] [05:21:58] <mato> we talked about the non-zero-copy APIs
[Monday 20 September 2010] [05:22:10] <mato> and i have the most obvious names :)
[Monday 20 September 2010] [05:22:22] <mato> zmq_sendcopy and zmq_recvcopy ... ?
[Monday 20 September 2010] [05:22:43] <mato> would do fine for 2.1, in 3.x the whole mess can be done the right way
[Monday 20 September 2010] [05:23:31] <sustrik> dunno, does it makes sense to introduce something that will be changed anyway?
[Monday 20 September 2010] [05:23:45] <sustrik> it’s just asking for problems
[Monday 20 September 2010] [05:23:50] <mato> why?
[Monday 20 September 2010] [05:24:41] <mato> well, one reason is that pieter uses all these “helper” functions in the user guide precisely for this reason
[Monday 20 September 2010] [05:25:07] <sustrik> so, we’ll have send, sendcopy and the helper function
[Monday 20 September 2010] [05:25:16] <sustrik> :)
[Monday 20 September 2010] [05:25:16] <mato> no, the helpers can go away
[Monday 20 September 2010] [05:25:36] <mato> check with pieter, but imo the reason those are there is becuase zerocopy in C is “too much typing” or something :)
[Monday 20 September 2010] [05:25:43] <mato> at least that’s what he told me
[Monday 20 September 2010] [05:25:54] <sustrik> anyway, first thing on this path is to defined the semantics for recvcopy when the buffer is no large enough
[Monday 20 September 2010] [05:26:06] * sustrik has to have a look at sctp
[Monday 20 September 2010] [05:27:10] <mato> yeah, good point
[Monday 20 September 2010] [05:27:28] <mato> presumably some error will have to be returned ...
[Monday 20 September 2010] [05:27:47] <mato> SCTP works in terms of messages as atomic units?
[Monday 20 September 2010] [05:29:31] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [05:30:28] <sustrik> mato: yes
[Monday 20 September 2010] [08:05:11] Quit mitkok has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [08:14:24] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [08:16:38] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 20 September 2010] [09:03:10] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [09:03:58] Quit dbudworth has left this server (Quit: dbudworth).
[Monday 20 September 2010] [09:25:09] Quit guido_g has left this server (Quit: Boom...).
[Monday 20 September 2010] [09:36:51] <user> was hoping someone can help me
[Monday 20 September 2010] [09:37:43] <user> cant download anything always comes up with an error
[Monday 20 September 2010] [09:44:18] <user> whats that?
[Monday 20 September 2010] [09:44:43] <drbobbeaty> The POSIX version is the version for Linux, etc. There’s also a Windows version on the same page.
[Monday 20 September 2010] [09:44:47] <mato> sustrik: are you around?
[Monday 20 September 2010] [09:45:20] <mato> sustrik: I’ve found a silly bug in the select() impl. of zmq_poll() on master
[Monday 20 September 2010] [09:45:31] <sustrik> mato: here am I
[Monday 20 September 2010] [09:45:35] <mato> sustrik: trivial fix, but something else is broken when using select()
[Monday 20 September 2010] [09:45:44] <user> k. my error code tells me: Archive: /media/OFFICE12/setup.exe
[Monday 20 September 2010] [09:45:44] <user> [/media/OFFICE12/setup.exe]
[Monday 20 September 2010] [09:45:44] <user> End-of-central-directory signature not found. Either this file is not
[Monday 20 September 2010] [09:45:44] <user> a zipfile, or it constitutes one disk of a multi-part archive. In the
[Monday 20 September 2010] [09:45:44] <user> latter case the central directory and zipfile comment will be found on
[Monday 20 September 2010] [09:45:45] <user> the last disk(s) of this archive.
[Monday 20 September 2010] [09:45:47] <user> note: /media/OFFICE12/setup.exe may be a plain executable, not an archive
[Monday 20 September 2010] [09:45:49] <user> zipinfo: cannot find zipfile directory in one of /media/OFFICE12/setup.exe or
[Monday 20 September 2010] [09:45:51] <user> /media/OFFICE12/setup.exe.zip, and cannot find /media/OFFICE12/setup.exe.ZIP, period.
[Monday 20 September 2010] [09:46:25] <mato> user: I’m sorry, you’re probably on the wrong chat room here.
[Monday 20 September 2010] [09:46:38] <user> where do i need to go?
[Monday 20 September 2010] [09:47:15] <sustrik> :)
[Monday 20 September 2010] [09:47:46] <sustrik> mato: wull?
[Monday 20 September 2010] [09:48:02] <mato> sustrik: wull?
[Monday 20 September 2010] [09:48:11] <sustrik> well?
[Monday 20 September 2010] [09:48:17] <mato> hang on
[Monday 20 September 2010] [09:48:20] <mato> i’m on bloody win32
[Monday 20 September 2010] [09:48:25] <mato> everything is confusing :-)
[Monday 20 September 2010] [09:48:28] <mato> patience...
[Monday 20 September 2010] [09:49:28] <mato> sustrik: ok, 1st, somewhee around line 547 of zmq.cpp, the select () call needs to be changed to use maxfd + 1
[Monday 20 September 2010] [09:49:32] <mato> sustrik: not just maxfd
[Monday 20 September 2010] [09:49:37] <mato> sustrik: that’s probably my fault
[Monday 20 September 2010] [09:49:43] <sustrik> ok
[Monday 20 September 2010] [09:50:26] <mato> sustrik: now, then, what i’m seeing is on win32 _or_ on Linux with ZMQ_FORCE_SELECT (and I patched zmq.cpp to also use select() for zmq_poll() when zmq_force_select() is defined)
[Monday 20 September 2010] [09:50:52] <mato> sustrik: for some reason a socket is not becoming ready on the app side when it should
[Monday 20 September 2010] [09:51:34] <mato> sustrik: i.e. data gets sent down XREP, XREQ on the client side never becomes ready
[Monday 20 September 2010] [09:51:58] <sustrik> do you have a simple test program?
[Monday 20 September 2010] [09:52:06] <mato> working on it...
[Monday 20 September 2010] [09:52:52] <sustrik> ok
[Monday 20 September 2010] [09:54:35] Part user has left this channel (“Leaving”).
[Monday 20 September 2010] [10:20:07] <mato> sustrik: ok, i have a test case... will msg you, it;s on the test box
[Monday 20 September 2010] [10:45:13] <CIA-20> zeromq2: 03Martin Lucina 07master * r1abfc92 10/ src/zmq.cpp : minor problem in zmq_poll (select version) fixed -
http://bit.ly/c9Sdnb[Monday 20 September 2010] [10:55:34] <CIA-20> zeromq2: 03Martin Lucina 07master * rf49b77e 10/ src/zmq.cpp : zmq_poll honours ZMQ_FORCE_POLL and ZMQ_FORCE_SELECT options -
http://bit.ly/dzi76e[Monday 20 September 2010] [10:56:23] Quit GeekGod has left this server (Quit: GeekGod).
[Monday 20 September 2010] [11:16:12] <sustrik> mato: afaics the problem is that ZMQ_FD is edge-trigerred
[Monday 20 September 2010] [11:17:26] <sustrik> thus, IN/OUT flag may be set in the past, but the select/poll won’t exit because of it
[Monday 20 September 2010] [11:18:32] <sustrik> mato: check how poll version of zmq_poll works
[Monday 20 September 2010] [11:20:32] <mato> sustrik: the code seems equivalent, no?
[Monday 20 September 2010] [11:20:36] <sustrik> nope
[Monday 20 September 2010] [11:21:02] <mato> then i don’t understand the problem...
[Monday 20 September 2010] [11:21:05] <sustrik> you have to delete lines 572.577
[Monday 20 September 2010] [11:21:11] <sustrik> let me do it
[Monday 20 September 2010] [11:21:58] <mato> which lines, i have different line numbers here...
[Monday 20 September 2010] [11:22:12] <mato> and i’d like to understand what the problem is
[Monday 20 September 2010] [11:22:46] <sustrik> you should _not_ check POLLIN is set on ZMQ_FD and check ZMQ_EVENTS anyway
[Monday 20 September 2010] [11:22:56] <sustrik> whather
[Monday 20 September 2010] [11:22:59] <sustrik> ehether
[Monday 20 September 2010] [11:23:01] <sustrik> whether
[Monday 20 September 2010] [11:23:02] <sustrik> :)
[Monday 20 September 2010] [11:23:03] <mato> ?
[Monday 20 September 2010] [11:23:16] <sustrik> ...whether POLLIN is set...
[Monday 20 September 2010] [11:23:48] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [11:24:44] <mato> sustrik: hmm, so you’re saying that each ZMQ_FD needs to be checked every time you come out of the select/poll() ?
[Monday 20 September 2010] [11:24:48] <CIA-20> zeromq2: 03Martin Sustrik 07master * r4d51a52 10/ src/zmq.cpp : zmq_poll (select version) now correctly assumes that ZMQ_FD is edge-trigerred -
http://bit.ly/dklauP[Monday 20 September 2010] [11:25:07] <sustrik> yes
[Monday 20 September 2010] [11:25:15] <sustrik> committed
[Monday 20 September 2010] [11:25:23] <sustrik> the test program seems to work
[Monday 20 September 2010] [11:26:23] <mato> i still don’t understand... select would not have exited if the fd did not become ready...?
[Monday 20 September 2010] [11:26:41] Quit jsimmons has left this server (Ping timeout: 276 seconds).
[Monday 20 September 2010] [11:27:26] <sustrik> what?
[Monday 20 September 2010] [11:27:51] <mato> sustrik: if you’re sitting in select(), and the notify fd becomes ready, then you read the events
[Monday 20 September 2010] [11:27:59] <mato> sustrik: what is the other code path?
[Monday 20 September 2010] [11:28:18] <sustrik> the commands in ZMQ_FD were already processed before calling zmq_poll
[Monday 20 September 2010] [11:28:25] <sustrik> select blocks forever
[Monday 20 September 2010] [11:28:38] <sustrik> although there are messages in available
[Monday 20 September 2010] [11:28:40] <mato> processed by who?
[Monday 20 September 2010] [11:28:51] <sustrik> random previous command
[Monday 20 September 2010] [11:29:12] <mato> ah, right, this is because ZMQ_FD is tapping straight into the signaller
[Monday 20 September 2010] [11:29:15] <mato> mumble
[Monday 20 September 2010] [11:29:20] <sustrik> ack
[Monday 20 September 2010] [11:29:22] <mato> i wish we could fix that
[Monday 20 September 2010] [11:29:27] <sustrik> ?
[Monday 20 September 2010] [11:29:33] <sustrik> it’s fixed
[Monday 20 September 2010] [11:29:42] <mato> this is also why you do that first_pass thing, right?
[Monday 20 September 2010] [11:29:47] <sustrik> yes
[Monday 20 September 2010] [11:29:55] <sustrik> no timout on first pass
[Monday 20 September 2010] [11:30:04] <sustrik> exit immediately
[Monday 20 September 2010] [11:30:15] <sustrik> then check whether events are availalbe
[Monday 20 September 2010] [11:30:25] <mato> this is to pick up events coming from previously processed commands, right?
[Monday 20 September 2010] [11:30:34] <sustrik> yes
[Monday 20 September 2010] [11:30:40] <mato> ok, understood
[Monday 20 September 2010] [11:31:00] * mato will re-test and see what happens
[Monday 20 September 2010] [11:38:39] <sustrik> mato: btw, if you want to optimise it you can still perform the check when !first_pass
[Monday 20 September 2010] [11:39:16] <sustrik> getting ZMQ_EVENTS can be rather slow as it involves reading from the signaler => recv()
[Monday 20 September 2010] [11:40:40] <mato> right, that might be a good idea
[Monday 20 September 2010] [11:40:48] <mato> also, is getting ZMQ_FD slow?
[Monday 20 September 2010] [11:41:01] <mato> since that is also done more times than strictly necessary
[Monday 20 September 2010] [11:41:13] <mato> sustrik: btw, the signaler uses socketpair?
[Monday 20 September 2010] [11:41:32] <mato> sustrik: which translates to a tcp connection with winsock?
[Monday 20 September 2010] [11:44:30] <sustrik> yes
[Monday 20 September 2010] [11:44:54] <mato> interesting, i get an occasional “Address already in use” on Win32 from signaler.cpp:80
[Monday 20 September 2010] [11:45:06] <sustrik> let me see
[Monday 20 September 2010] [11:45:18] <mato> I guess this is just the poor M$ WIn$ock running out of tcp ports or something
[Monday 20 September 2010] [11:45:27] <mato> it’s the “connect to remote peer” call...
[Monday 20 September 2010] [11:45:34] <sustrik> EADDRINUSE on connect?
[Monday 20 September 2010] [11:45:52] <mato> yeah
[Monday 20 September 2010] [11:45:56] <mato> bizarre
[Monday 20 September 2010] [11:46:04] <sustrik> hm, it’s documented
[Monday 20 September 2010] [11:46:16] <mato> “not enough ports” ?
[Monday 20 September 2010] [11:47:01] <sustrik> very useful description:
[Monday 20 September 2010] [11:47:02] <sustrik> [EADDRINUSE]
[Monday 20 September 2010] [11:47:02] <sustrik> Attempt to establish a connection that uses addresses that are already in use.
[Monday 20 September 2010] [11:47:30] Quit xla has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [11:47:33] <sustrik> (that’s POSIX)
[Monday 20 September 2010] [11:48:00] <mato> sustrik: windoze is slightly different
[Monday 20 September 2010] [11:48:05] <sustrik> Linux: EADDRINUSE
[Monday 20 September 2010] [11:48:05] <sustrik> Local address is already in use.
[Monday 20 September 2010] [11:49:17] <sustrik> The socket’s local address is already in use and the socket was not marked to allow address reuse with SO_REUSEADDR. This error usually occurs when executing bind, but could be delayed until the connect function if the bind was to a wildcard address (INADDR_ANY or in6addr_any) for the local IP address. A specific address needs to be implicitly bound by the connect function.
[Monday 20 September 2010] [11:49:40] <mato> Yeah, I never quite understood the SO_REUSEADDR semantics on win32
[Monday 20 September 2010] [11:49:47] <sustrik> does it make any sense to you?
[Monday 20 September 2010] [11:49:54] <mato> but i’ll try adding that in and see if it changes anything
[Monday 20 September 2010] [11:50:19] <mato> well, i think what it’s saying is “the wildcard bind() picked a port that is still in time-wait state”
[Monday 20 September 2010] [11:50:27] <mato> or some nonsense like that
[Monday 20 September 2010] [11:50:31] <sustrik> :|
[Monday 20 September 2010] [11:50:42] <mato> let me try adding SO_REUSEADDR for win32 to the signaler listen socket and see what happens
[Monday 20 September 2010] [11:50:57] <mato> i don’t care what it actually does on win32 as long as the error goes away :-)
[Monday 20 September 2010] [11:51:25] <mato> proper win32 solution is obviously to use named pipes/win32 objects/whatsits and iocp
[Monday 20 September 2010] [11:54:09] <sustrik> ok
[Monday 20 September 2010] [12:07:35] <mato> hmm
[Monday 20 September 2010] [12:07:38] <mato> i dunno
[Monday 20 September 2010] [12:07:42] <mato> doesn’t seem to help much
[Monday 20 September 2010] [12:07:47] <mato> anyway, this doesn’t really matter
[Monday 20 September 2010] [12:08:11] <mato> i added SO_REUSEADDR to both ends of the emulated socketpair and still get EADDRINUSE back
[Monday 20 September 2010] [12:08:27] <mato> so i think it’s just poor windows running out of ports to auto-assign or something :-)
[Monday 20 September 2010] [12:08:50] <mato> sustrik: anyway, the problem with select() has been fixed, so it’s all good!
[Monday 20 September 2010] [12:11:05] <sustrik> nice
[Monday 20 September 2010] [12:11:26] <sustrik> you may fill in the bug report for the EADDRINUSE stuff
[Monday 20 September 2010] [12:24:30] <mato> done
[Monday 20 September 2010] [12:45:23] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Monday 20 September 2010] [13:07:13] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Monday 20 September 2010] [13:14:11] Quit ModusPwnens has left this server (Ping timeout: 255 seconds).
[Monday 20 September 2010] [13:18:32] Quit psino has left this server (Quit: psino).
[Monday 20 September 2010] [13:38:12] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Monday 20 September 2010] [13:50:52] Quit mivert has left this server (Quit: Leaving).
[Monday 20 September 2010] [14:28:40] Quit omarkj has left this server (Quit: omarkj).
[Monday 20 September 2010] [14:52:12] Quit mitkok has left this server (Read error: No route to host).
[Monday 20 September 2010] [14:55:48] Quit mitkok1 has left this server (Client Quit).
[Monday 20 September 2010] [15:13:59] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 20 September 2010] [15:54:16] Quit kelvink has left this server (Quit: Leaving.).
[Monday 20 September 2010] [16:00:52] <cpscotti> Hello there, anyone up to a philosophical (although basic) discussion on a zmq networking topology for a “generic” application? Regarding many clients & services but without a broker.
[Monday 20 September 2010] [16:03:43] Quit jhawk28 has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [16:07:18] Quit mitkok has left this server (Client Quit).
[Monday 20 September 2010] [16:08:37] Nick mitkok is now known as mytrile.
[Monday 20 September 2010] [16:17:01] <cremes> cpscotti: i’d be happy to have that conversation with you tomorrow; i have to leave for the rest of today
[Monday 20 September 2010] [16:19:20] <cpscotti> cremes: thanks.. if nothing is solved in my side of things I’ll try tomorrow then
[Monday 20 September 2010] [16:19:50] <cremes> ok
[Monday 20 September 2010] [16:32:22] Quit mytrile has left this server (Read error: No route to host).
[Monday 20 September 2010] [16:34:27] Quit mitkok has left this server (Client Quit).
[Monday 20 September 2010] [17:07:39] <ModusPwnens> is the send function faster than the receive function? or are they both just as fast?
[Monday 20 September 2010] [17:08:10] <cpscotti> send doesn’t block, recv blocks
[Monday 20 September 2010] [17:08:21] <cpscotti> (as for the “program flow” speed)
[Monday 20 September 2010] [17:08:33] <cpscotti> now for the underlying stuff, dunno
[Monday 20 September 2010] [17:08:36] <ModusPwnens> even with pub/sub topology?
[Monday 20 September 2010] [17:08:37] <jhawk28> recv can do a noblock flag
[Monday 20 September 2010] [17:08:49] <cpscotti> awl.. yep..
[Monday 20 September 2010] [17:08:56] <ModusPwnens> I see.
[Monday 20 September 2010] [17:09:09] <ModusPwnens> So that would explain benchmarking tests where
[Monday 20 September 2010] [17:09:12] <ModusPwnens> i run several in a row
[Monday 20 September 2010] [17:09:59] <ModusPwnens> thats probably better than explaining it
[Monday 20 September 2010] [17:10:37] <ModusPwnens> I’ve sort of run into a wall with this and have been stuck on it for several days
[Monday 20 September 2010] [17:10:44] <ModusPwnens> because I am seeing those results
[Monday 20 September 2010] [17:10:47] <ModusPwnens> and am not sure what to make of them
[Monday 20 September 2010] [18:10:12] Quit
antares_ has left this server (Quit:
antares_).
[Monday 20 September 2010] [18:26:19] Quit cpscotti has left this server (Ping timeout: 252 seconds).
[Monday 20 September 2010] [18:30:32] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 20 September 2010] [18:35:19] Quit mytrile has left this server (Quit: Leaving.).
[Monday 20 September 2010] [18:41:15] <larrytheliquid> are there any semantic implications to connecting multiple times to the same address?
[Monday 20 September 2010] [18:41:36] <larrytheliquid> from the same socket, that is
[Monday 20 September 2010] [18:44:18] Quit sbahra has left this server (Remote host closed the connection).
[Monday 20 September 2010] [18:45:20] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 20 September 2010] [18:49:06] Quit gavinstark has left this server (Remote host closed the connection).
[Monday 20 September 2010] [18:53:43] Quit dbudworth has left this server (Quit: dbudworth).
[Monday 20 September 2010] [19:02:09] Quit ModusPwnens has left this server.
[Monday 20 September 2010] [19:31:31] Quit Samy has left this server (Ping timeout: 240 seconds).
[Monday 20 September 2010] [19:36:02] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Monday 20 September 2010] [19:38:33] Join Samy has joined this channel (
~sbahra@8.8.38.2).
[Monday 20 September 2010] [20:58:34] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Monday 20 September 2010] [21:17:48] Quit andrewvc has left this server (Quit: andrewvc).
*** on Mon Sep 20 21:52:23 2010
[Monday 20 September 2010] [21:52:23] Topic The channel topic is “Welcome!”.
[Monday 20 September 2010] [21:52:27] Mode Channel modes: no messages from outside, topic protection
[Monday 20 September 2010] [21:52:27] Created This channel was created on 2010-02-05 10:44.
[Monday 20 September 2010] [22:15:40] Quit jhawk28 has left this server (Remote host closed the connection).
[Monday 20 September 2010] [23:10:53] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 21 September 2010] [00:22:53] Quit iFire has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [01:29:13] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [01:37:27] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 21 September 2010] [01:43:53] Quit RobertLJ has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [01:45:30] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Tuesday 21 September 2010] [01:55:18] Quit grahamalot has left this server (Quit: grahamalot).
[Tuesday 21 September 2010] [03:03:31] Quit benford has left this server (Quit: benford).
[Tuesday 21 September 2010] [03:04:26] Quit kleppari has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [03:10:35] Join kleppari has joined this channel (
~spa@bitbucket.is).
[Tuesday 21 September 2010] [03:36:36] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [04:05:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * r9c11886 10/ (src/socket_base.cpp src/socket_base.hpp): The flag in the socket has clear semantics now – it tracks whether corresponding context was closed, it doesn’t track whether zmq_close was called on the socket itself -
http://bit.ly/d6ktZI[Tuesday 21 September 2010] [04:05:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * re478468 10/ src/socket_base.hpp : minor error in comment fixed -
http://bit.ly/a4RuQJ[Tuesday 21 September 2010] [04:45:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [04:50:56] Quit xla has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [04:52:20] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [05:20:42] Quit xla has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [05:22:11] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 21 September 2010] [05:25:11] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [06:37:38] Quit SecretAgent has left this server (Ping timeout: 276 seconds).
[Tuesday 21 September 2010] [07:19:50] Quit timothyfitz has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [08:52:37] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [09:38:31] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [10:30:52] Quit mytrile has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [10:33:26] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 21 September 2010] [11:23:17] Part sd88g93 has left this channel.
[Tuesday 21 September 2010] [11:56:30] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [11:56:52] <starkdg> hello
[Tuesday 21 September 2010] [11:57:09] <guido_g> hi
[Tuesday 21 September 2010] [12:12:06] <CIA-20> rbzmq: 03Brian Buchanan 07master * rdc552c2 10/ rbzmq.c : Make PUSH and PULL constants work even if using an older version of the library. -
http://bit.ly/9ZisAu[Tuesday 21 September 2010] [12:12:08] <CIA-20> rbzmq: 03Brian Buchanan 07master * rf17f6ca 10/ zmq.gemspec : update gem version to 2.0.9 -
http://bit.ly/b4wU0A[Tuesday 21 September 2010] [12:21:58] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [13:02:20] Quit RobertLJ has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [13:24:48] Quit mytrile has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [13:53:42] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 21 September 2010] [13:56:46] Quit starkdg has left this server (Quit: goodbye).
[Tuesday 21 September 2010] [14:05:19] Quit xla has left this server (Read error: Operation timed out).
[Tuesday 21 September 2010] [14:19:27] <AltraMayor> Hi there,
[Tuesday 21 September 2010] [14:20:06] <AltraMayor> I’ve been reading about PNUTS, a NoSQL solution written by Yahoo!
[Tuesday 21 September 2010] [14:21:21] <AltraMayor> A very important element of their design is their message brokers, which guarantee that messages aren’t lost. For example, they can make brokers replicate messages with others to support failure.
[Tuesday 21 September 2010] [14:21:45] <AltraMayor> Is this use case supported by zeromq?
[Tuesday 21 September 2010] [14:21:55] <guido_g> no, zeromq doesn’t have that
[Tuesday 21 September 2010] [14:22:34] <AltraMayor> Thanks, guido_g.
[Tuesday 21 September 2010] [14:23:21] <AltraMayor> I’ve read that page, but since it says that’s more coming, I couldn’t assume that it’s all.
[Tuesday 21 September 2010] [14:23:54] <AltraMayor> Does anyone know a message broker designed to support that level of failure?
[Tuesday 21 September 2010] [14:24:09] <guido_g> you m ight have skipped the important things
[Tuesday 21 September 2010] [14:24:31] <AltraMayor> I don’t follow you, guido_g.
[Tuesday 21 September 2010] [14:24:33] <guido_g> like that mq doesn’t have or is a message broker
[Tuesday 21 September 2010] [14:25:11] <AltraMayor> I understand that zeromq isn’t a broker, but its durable option isn’t enough.
[Tuesday 21 September 2010] [14:25:41] <cremes> AltraMayor: the roadmap is a wiki page that can be edited by anyone who signs up for an account
[Tuesday 21 September 2010] [14:25:52] <cremes> if you think there are things missing, feel free to add them
[Tuesday 21 September 2010] [14:25:55] <AltraMayor> I also understood I could implement a broker as reliable as I want using zeromq, but this would be just a piece of what I’m looking for.
[Tuesday 21 September 2010] [14:29:12] <AltraMayor> Cremes: what’s the link to the roadmap page you mentioned?
[Tuesday 21 September 2010] [14:31:55] <AltraMayor> Cremes: do you think there’s interest among the developers to implement this level of robustness? As I read, zeroqm is implemented by iMatix to meet their demands, I don’t know if they’d be interested in a wish list of a guy that wants to have an open-source alternative to PNUTS.
[Tuesday 21 September 2010] [14:32:25] <cremes> good point... i heard their ceo speak on that very topic yesterday
[Tuesday 21 September 2010] [14:32:34] <cremes> if you write the code, it will get done
[Tuesday 21 September 2010] [14:32:41] <cremes> if you pay them to write the code, it will get done
[Tuesday 21 September 2010] [14:35:03] <AltraMayor> I understand, but I’m looking for pieces to estimate the total effort to have an opensource PNUTS.
[Tuesday 21 September 2010] [14:36:37] <cremes> okay, well i don’t know what pnuts is :)
[Tuesday 21 September 2010] [14:39:57] <AltraMayor> Cremes: can you provide me a small description how you use 0mq?
[Tuesday 21 September 2010] [14:41:04] <cremes> AltraMayor: sure...
[Tuesday 21 September 2010] [14:41:24] <cremes> i use it to tie together a bunch of distributed components; i used to use amqp but threw it away for 0mq
[Tuesday 21 September 2010] [14:41:40] <cremes> i do some pub/sub and some request/reply style message passing
[Tuesday 21 September 2010] [14:41:55] <cremes> that’s the short version
[Tuesday 21 September 2010] [14:42:01] <AltraMayor> How do your components deal with failure? Do you just restart them?
[Tuesday 21 September 2010] [14:42:27] <cremes> it depends
[Tuesday 21 September 2010] [14:42:45] <cremes> if certain components fail, it is fatal for the whole system
[Tuesday 21 September 2010] [14:42:58] <AltraMayor> By the way, PNUTS is a NoSQL that supports replication/accesses/updates among datacenters.
[Tuesday 21 September 2010] [14:43:05] <cremes> if others fail, i detect their absence and just restart/resend their jobs to surviving nodes
[Tuesday 21 September 2010] [14:43:32] <cremes> then perhaps you want to look at mongodb as something similar; 0mq has nothing to do with nosql
[Tuesday 21 September 2010] [14:43:40] <cremes> it’s not a database in any way, shape or form
[Tuesday 21 September 2010] [14:47:47] <AltraMayor> I see that 0mq isn’t a database, but messaging brokers are key in PNUTS’ design. Mongodb is the best NoSQL that I know so far, but it don’t support datacenters as graceful as PNUTS. And, I’m considering implement some pieces, I just didn’t want to start from so low as the message broker.
[Tuesday 21 September 2010] [14:48:36] <guido_g> then get something w/ a broker
[Tuesday 21 September 2010] [14:49:36] <AltraMayor> Ok.
[Tuesday 21 September 2010] [14:50:08] <AltraMayor> Thank you for your time!
[Tuesday 21 September 2010] [14:50:39] Quit AltraMayor has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [14:50:52] <guido_g> best would be broker which stores the messages in a reliable way distributed among datacenters
[Tuesday 21 September 2010] [14:51:28] <guido_g> now wating for the stack overflow...
[Tuesday 21 September 2010] [15:30:36] Quit cpscotti has left this server (Ping timeout: 252 seconds).
[Tuesday 21 September 2010] [15:49:18] Quit mytrile has left this server (Quit: Leaving.).
[Tuesday 21 September 2010] [16:25:49] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 21 September 2010] [16:56:11] <Samy> like the new website
[Tuesday 21 September 2010] [16:56:49] <Samy> It would be nice if there was a more obvious link to the source repository.
[Tuesday 21 September 2010] [16:59:06] <Samy> Ah, zeromq.com != zeromq.org
[Tuesday 21 September 2010] [17:07:55] <mato> Samy: yeah, the .com site is intended as a portal for new users
[Tuesday 21 September 2010] [17:08:09] <mato> Samy: those who just want a first quick look
[Tuesday 21 September 2010] [17:11:21] <jond> mato: hi
[Tuesday 21 September 2010] [17:14:31] <mato> jond: hi there
[Tuesday 21 September 2010] [17:15:04] <jond> mato: i’ve been looking at the select.cpp and i think there is an issue
[Tuesday 21 September 2010] [17:15:23] <jond> because I’m fixing something similar in the dayjob
[Tuesday 21 September 2010] [17:17:08] <mato> jond: what kind of issue?
[Tuesday 21 September 2010] [17:17:14] <jond> as well as checking that the array doesnt get to FD_SETSIZE, it needs to check that the
fd_ in add_fd is actually greater than FD_SETSIZE
[Tuesday 21 September 2010] [17:18:45] <mato> why would the actual fd matter?
[Tuesday 21 September 2010] [17:18:49] <jond> FD_SET, CLR etc don’t work on linux above FD_SETSIZE. it’s on the man page at the bottom; I had a process with 6000 fd’s open and once the sockets started connecting and getting fd > 1024 bad things happen
[Tuesday 21 September 2010] [17:19:31] <jond> fd_set is normally implemented as a bitvector of fixed size
[Tuesday 21 September 2010] [17:19:33] <mato> ah, you’re right
[Tuesday 21 September 2010] [17:19:35] <mato> yeah
[Tuesday 21 September 2010] [17:19:46] <mato> good point
[Tuesday 21 September 2010] [17:20:21] <mato> well, the select implementation has not really been stress-tested properly
[Tuesday 21 September 2010] [17:20:40] <mato> it’s only there for windows benefit (and vms I guess)
[Tuesday 21 September 2010] [17:20:52] <jond> we raised call with redhat and even if you recompile glibc etc the kernel has it at 1024. My problem is that the 6000 fd’s are files opened early and the sockets connect later and get to high fd.
[Tuesday 21 September 2010] [17:21:02] <jond> i’m changing to poll
[Tuesday 21 September 2010] [17:21:27] <mato> well, if you’re dealing with 000’s of fds you really should not be using select at all :-)
[Tuesday 21 September 2010] [17:21:28] <jond> on solaris I believe you can just redefine FD_SETSIZE
[Tuesday 21 September 2010] [17:21:48] <mato> and poll is not too performant with lots of fds either, you really want an edge-triggered interface
[Tuesday 21 September 2010] [17:22:27] <mato> anyhow, thanks for the tip; i’ll make a note to make a patch that adds in the appropriate assertions
[Tuesday 21 September 2010] [17:23:03] <jond> yep I know, but I ‘inherited’ this code; we only have a 600 tcp connections, but open lots of files....
[Tuesday 21 September 2010] [17:24:38] <jond> mato: edge-triggered, you mean epoll?
[Tuesday 21 September 2010] [17:27:02] <mato> jond: yes
[Tuesday 21 September 2010] [17:27:21] <mato> jond: unfortunately epoll never got standardised so is available only on Linux
[Tuesday 21 September 2010] [17:27:42] <mato> jond:
*BSD has kqueue, Solaris has Event Completion
[Tuesday 21 September 2010] [17:27:53] <jond> i’ll take a look tommorrow; thanks for the tip;
[Tuesday 21 September 2010] [17:28:09] <jond> mato: cheers
[Tuesday 21 September 2010] [17:36:42] <bgranger> mato: did you see my reply about the message related deadlock you were seeing?
[Tuesday 21 September 2010] [17:38:31] <jond> bgranger: i did. is there a link to that pyzmq branch
[Tuesday 21 September 2010] [17:38:39] <bgranger> Sure, right here...
[Tuesday 21 September 2010] [17:38:54] <dermoth> Is there any example of XREQ/XREP out there? I’ve used REP/REP successfully in the past and now I’d like to add devices in the middle, and I believe I need XREQ/XREP for that.
[Tuesday 21 September 2010] [17:39:11] <bgranger> Here is the class
[Tuesday 21 September 2010] [17:40:03] <bgranger> We are still doing code review so a few more things might change in the API, but I think things are mostly in place.
[Tuesday 21 September 2010] [17:40:49] <mato> bgranger: hi
[Tuesday 21 September 2010] [17:41:04] <bgranger> hi
[Tuesday 21 September 2010] [17:41:12] <mato> bgranger: yes, i saw that, am travelling most of this week (Linux Kongress in Nurnberg, Martin Sustrik is presenting)
[Tuesday 21 September 2010] [17:41:21] <bgranger> Nice
[Tuesday 21 September 2010] [17:41:30] <jond> bgranger: thanks, i’ll take a look. I think that function that’s added in py2.7 for PendingCall looks interesting and could still provide a version which doesnt need the MesssageTrcaker
[Tuesday 21 September 2010] [17:41:33] <mato> bgranger: will look at it next week once I get back home
[Tuesday 21 September 2010] [17:41:48] <bgranger> Great, sounds good, by then it will probably be in our master.
[Tuesday 21 September 2010] [17:41:50] <mato> bgranger: can’t really test it reliably while on the road.
[Tuesday 21 September 2010] [17:41:59] <bgranger> OK
[Tuesday 21 September 2010] [17:53:17] Quit mytrile has left this server (Read error: Connection reset by peer).
*** on Tue Sep 21 17:58:29 2010
[Tuesday 21 September 2010] [17:58:29] Topic The channel topic is “Welcome!”.
[Tuesday 21 September 2010] [17:58:33] Mode Channel modes: no messages from outside, topic protection
[Tuesday 21 September 2010] [17:58:33] Created This channel was created on 2010-02-05 10:44.
[Tuesday 21 September 2010] [17:58:43] <dermoth> [bbl]
[Tuesday 21 September 2010] [18:44:02] Quit jhawk28 has left this server (Ping timeout: 265 seconds).
[Tuesday 21 September 2010] [18:51:17] Quit mytrile has left this server (Ping timeout: 265 seconds).
[Tuesday 21 September 2010] [19:27:45] <tylergillies> whats the difference between req/rep and upstream/downstream?
[Tuesday 21 September 2010] [19:31:11] <vanadium> req/rep are bidirectional/back and forth, upstream/downstream are push/pull and one of them always sends and the other always receives
[Tuesday 21 September 2010] [19:38:36] Quit dbudworth has left this server (Quit: dbudworth).
[Tuesday 21 September 2010] [19:46:19] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 21 September 2010] [20:27:15] Part starkdg has left this channel.
[Tuesday 21 September 2010] [20:32:04] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Tuesday 21 September 2010] [20:43:19] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [20:59:19] Quit
jhawk28_ has left this server (Read error: Connection reset by peer).
[Tuesday 21 September 2010] [21:03:26] <dermoth> actually one of the issues that I was seeing is obvious... in a php worker I want to have a persistent socket that use req/rep (or the x equivalent if needed), but if the backend isn’T responding timely I just want to leave early and go on, but then when the next request comes in the socket is still expecting a reply... how can I work around this?
[Tuesday 21 September 2010] [21:03:41] <dermoth> i guess that’S a good one for the ML :)
[Tuesday 21 September 2010] [21:05:59] Quit andrewvc has left this server (Quit: andrewvc).
[Tuesday 21 September 2010] [21:18:31] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Tuesday 21 September 2010] [22:10:36] Quit jhawk28 has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [00:01:14] Quit RobertLJ has left this server (Ping timeout: 264 seconds).
[Wednesday 22 September 2010] [00:19:54] Quit kleppari has left this server (Ping timeout: 272 seconds).
*** on Wed Sep 22 07:12:46 2010
[Wednesday 22 September 2010] [07:12:46] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [07:12:50] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [07:12:50] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [07:53:53] <raz> hm
[Wednesday 22 September 2010] [07:53:58] <raz> looking at the divide&conquer example i wonder
[Wednesday 22 September 2010] [07:54:18] <raz> how does zmq behave when a worker does not indicate it finished? (e.g. because it crashed)
[Wednesday 22 September 2010] [07:54:25] <raz> will the task be re-queued to a different worker?
[Wednesday 22 September 2010] [08:00:10] Notice -NickServ- This nickname is registered. Please choose a different nickname, or identify via /msg NickServ identify <password>.
[Wednesday 22 September 2010] [08:00:10] Notice -NickServ- You are now identified for travlr.
[Wednesday 22 September 2010] [08:00:10] CTCP Received Version request from frigg.
[Wednesday 22 September 2010] [08:00:14] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [08:00:18] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [08:00:18] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [08:02:12] <guido_g> no, you have to do it yourself
[Wednesday 22 September 2010] [08:07:53] <raz> hm
[Wednesday 22 September 2010] [08:08:36] <raz> the taskvent example doesn’t seem to inspect the return value at all
[Wednesday 22 September 2010] [08:08:53] <raz> would s_send return an error if no worker picks the message up?
[Wednesday 22 September 2010] [08:09:27] * raz tries to find some semantic overview of the different modes/types
[Wednesday 22 September 2010] [08:10:37] <raz> the zmq_send man-page does not mention such an error tho
[Wednesday 22 September 2010] [08:11:19] <guido_g> there should be a description what happens for the socket types if something goes wrong
[Wednesday 22 September 2010] [08:11:51] * guido_g needs an new english grammar module though
[Wednesday 22 September 2010] [08:12:20] <raz> hmyea
[Wednesday 22 September 2010] [08:12:30] <raz> the ZMQ_PUSH section doesn’t tell me what happens when a msg can not be delivered tho
[Wednesday 22 September 2010] [08:12:41] <raz> or rather.. i assumed there was some sort of acknowledgement going on
[Wednesday 22 September 2010] [08:12:45] <guido_g> nothing
[Wednesday 22 September 2010] [08:12:46] <raz> which is apparently on the case
[Wednesday 22 September 2010] [08:13:19] <raz> s/on/not/
[Wednesday 22 September 2010] [08:14:02] <guido_g> there is no guarantee for delivery in mq
[Wednesday 22 September 2010] [08:15:03] <raz> hmm
[Wednesday 22 September 2010] [08:30:56] Quit xla has left this server (Quit: leaving).
[Wednesday 22 September 2010] [08:37:26] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 22 September 2010] [08:37:48] <ajsie> zeromq doesnt support persistence?
[Wednesday 22 September 2010] [08:50:47] Quit xla has left this server (Quit: leaving).
[Wednesday 22 September 2010] [08:57:32] Quit sbahra has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [09:15:01] <guido_g> it can store messages, but that’s not what you know from broker based solutions
[Wednesday 22 September 2010] [09:23:58] Quit zsxxsz has left this server (Quit: Page closed).
[Wednesday 22 September 2010] [09:28:49] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 22 September 2010] [09:32:55] <keffo> mm, nice.. code dated 3/11/78
[Wednesday 22 September 2010] [09:37:24] <vanadium> If I have one push sockets sending workloads to a number of threads with pull sockets, the work will be evenly distributed over all threads even if some of them take way longer per workload, right?
[Wednesday 22 September 2010] [09:37:38] <vanadium> It is just a round-robin thing and not actually communicating back who is ready for more work or anything?
[Wednesday 22 September 2010] [09:50:11] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 22 September 2010] [09:50:42] <drbobbeaty> vanadium: I’ve not used the PUSH/PULL sockets before, but I believe that you are correct in your assumption. You have a single socket (PUSH) that’s bound to some URL, and a series of “worker threads” that each has a socket (PULL) that’s connected to that same URL. Then, as the single “producer” socket sends out messages, they are picked up and worked on, in turn, by the worker threads.
[Wednesday 22 September 2010] [09:51:24] <drbobbeaty> As i understand it, there are several ways to handle the distribution - round-robin, least-recently-used, etc., but I’m not as familiar with those concepts or how to set them up.
[Wednesday 22 September 2010] [09:51:24] <vanadium> Thank you
[Wednesday 22 September 2010] [09:51:58] <drbobbeaty> no problem
[Wednesday 22 September 2010] [09:58:09] Part ajsie has left this channel.
[Wednesday 22 September 2010] [10:03:28] <guido_g> re
[Wednesday 22 September 2010] [10:13:17] <raz> has anyone used zmq with ruby eventmachine?
[Wednesday 22 September 2010] [10:13:20] <raz> does it blend?
[Wednesday 22 September 2010] [10:17:39] <cremes> raz: the 2.0.x branch doesn’t blend well with EM; the 2.1.x branch might
[Wednesday 22 September 2010] [10:18:10] <raz> well, i have an existing em app that i’d like to add zmq to
[Wednesday 22 September 2010] [10:18:30] <raz> zmqmachine looks like it’s meant to be a replacement for em
[Wednesday 22 September 2010] [10:18:41] <cremes> right
[Wednesday 22 September 2010] [10:19:01] <cremes> you’d have to wire the 0mq sockets into the EM reactor loop
[Wednesday 22 September 2010] [10:19:09] <cremes> that isn’t possible with any 0mq release right now
[Wednesday 22 September 2010] [10:19:25] <raz> hrm
[Wednesday 22 September 2010] [10:19:28] <cremes> the 2.1.x branch has a few new file descriptor elements exposed which will make it feasible to do so
[Wednesday 22 September 2010] [10:19:43] <raz> any dirty trick possible with a thread or so?
[Wednesday 22 September 2010] [10:19:55] <cremes> of course, but that’s cheating :)
[Wednesday 22 September 2010] [10:19:59] <raz> hehe
[Wednesday 22 September 2010] [10:20:11] <raz> i don’t need it super pretty or fast, but super reliable would be nice ;)
[Wednesday 22 September 2010] [10:21:05] <raz> sadly my eventmachine fu isn’t all that advanced, not sure if i’d get the dispatching between eventmachine and a separate thread right
[Wednesday 22 September 2010] [10:21:06] <cremes> you would need to run the 0mq stuff in a thread separate from EM and any interaction would need to be scheduled via EM.schedule
[Wednesday 22 September 2010] [10:21:23] <cremes> 0mq -> EM would be via EM.schedule
[Wednesday 22 September 2010] [10:21:33] <cremes> EM -> 0mq could be handled straight up
[Wednesday 22 September 2010] [10:22:04] <raz> except when the 0mq call blocks (in the latter case)
[Wednesday 22 September 2010] [10:22:16] <raz> guess i should use a queue both ways
[Wednesday 22 September 2010] [10:22:23] <cremes> raz: you’d have to pass ZM_NOBLOCK to all send/recv calls
[Wednesday 22 September 2010] [10:22:48] <raz> hmm
[Wednesday 22 September 2010] [10:23:15] <cremes> you could maybe run zmqmachine and EM in parallel; all of that NO_BLOCK stuff is handled for you then
[Wednesday 22 September 2010] [10:23:18] <raz> sounds like that may be worth a shot
[Wednesday 22 September 2010] [10:23:25] <cremes> let us know how it turns out
[Wednesday 22 September 2010] [11:50:05] Quit dbudworth has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [11:53:39] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 22 September 2010] [12:48:09] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [12:54:38] Quit plol has left this server (Ping timeout: 264 seconds).
[Wednesday 22 September 2010] [12:57:31] Quit keffo has left this server (Ping timeout: 255 seconds).
[Wednesday 22 September 2010] [13:35:07] Part dbudworth has left this channel.
[Wednesday 22 September 2010] [13:35:39] Join plq has joined this channel (
~plq@88.249.173.198).
[Wednesday 22 September 2010] [13:36:58] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [13:45:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [13:45:56] Quit plq has left this server (Remote host closed the connection).
[Wednesday 22 September 2010] [13:49:42] <mankins> hi–i’ve got an installation problem with 2.0.9 under linux. configure: error: cannot link with -luuid, install uuid-dev.
[Wednesday 22 September 2010] [13:49:58] <mankins> only I’ve already got uuid-devel installed (under fedora core 8 on ec2)
[Wednesday 22 September 2010] [13:50:39] <mankins> libuuid.so.1 is at /lib ... not sure where to go next. Does this ring any bells?
[Wednesday 22 September 2010] [13:54:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [13:56:58] * mankins looks to mirror: oh! ln -s /usr/lib/libuuid.so.1 /usr/lib/libuuid.so fixed it.
[Wednesday 22 September 2010] [14:14:59] * mankins oh, maybe e2fsprogs-devel as well.
[Wednesday 22 September 2010] [14:18:21] Quit keffo has left this server (Ping timeout: 265 seconds).
[Wednesday 22 September 2010] [14:34:10] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [14:39:22] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [14:44:52] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [15:02:18] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 22 September 2010] [16:05:41] Quit xla has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [16:08:45] Quit
antares_ has left this server (Quit:
antares_).
[Wednesday 22 September 2010] [16:29:17] <CIA-20> jzmq: 03Stefan Majer 07master * rf3aa751 10/ (jzmq.spec pom.xml): RPM Packaging and Mavenized, first step. -
http://bit.ly/a6fxhX[Wednesday 22 September 2010] [17:07:10] <cremes> i’m wondering if anyone has seen an assertion failure in 2.0.9
[Wednesday 22 September 2010] [17:07:11] <cremes> Assertion failed: !engine (session.cpp:287)
[Wednesday 22 September 2010] [17:07:49] <cremes> i’m hitting that a lot but i can’t figure out why; i’m having a hard time reducing the code to a small example
[Wednesday 22 September 2010] [17:11:54] <pieterh> cremes: that’s a reported issue afair
[Wednesday 22 September 2010] [17:12:11] <pieterh> it’s caused by... /me vaguely recalls... identity crisis
[Wednesday 22 September 2010] [17:12:23] * pieterh goes to check issue tracker
[Wednesday 22 September 2010] [17:12:27] <cremes> ah... 2+ sockets with same identity...?
[Wednesday 22 September 2010] [17:12:37] <pieterh> yeah
[Wednesday 22 September 2010] [17:12:45] <pieterh> evil twin syndrome
[Wednesday 22 September 2010] [17:12:47] <cremes> cool, then i know where my bug is.... :)
[Wednesday 22 September 2010] [17:13:00] <pieterh> issue 30
[Wednesday 22 September 2010] [17:13:23] <pieterh> hey chuck, Chicago rocks :-)
[Wednesday 22 September 2010] [17:13:59] <cremes> i’m glad you had a good time... may i assume the trip was worthwhile from both a pleasure and business perspective?
[Wednesday 22 September 2010] [17:14:43] * pieterh is typing with his 1-month son in his arms... slightly tricky
[Wednesday 22 September 2010] [17:14:49] <cremes> heh
[Wednesday 22 September 2010] [17:15:07] <pieterh> 0mq + chicago = boom!
[Wednesday 22 September 2010] [17:15:21] <cremes> good boom or bad boom? ;)
[Wednesday 22 September 2010] [17:15:32] <pieterh> the good kind :-)
[Wednesday 22 September 2010] [17:15:50] <cremes> great... next time you come out bring the team
[Wednesday 22 September 2010] [17:16:24] <cremes> oops, gotta run; catch ya later!
[Wednesday 22 September 2010] [17:17:32] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 22 September 2010] [17:17:56] <pieterh> cyl
[Wednesday 22 September 2010] [17:21:10] <mato> pieterh: yo
[Wednesday 22 September 2010] [17:21:18] <mato> pieterh: just got back from dinner, etc with sustrik
[Wednesday 22 September 2010] [17:21:38] <mato> pieterh: linux-kongress starts tomorrow, martin is speaking around 5pm
[Wednesday 22 September 2010] [17:24:00] <pieterh> hey mato :-)
[Wednesday 22 September 2010] [17:24:11] <pieterh> how’s nurnberg?
[Wednesday 22 September 2010] [17:24:21] <mato> nice
[Wednesday 22 September 2010] [17:24:27] <mato> good beer
[Wednesday 22 September 2010] [17:24:35] <mato> some kind of local beer fest is on
[Wednesday 22 September 2010] [17:24:39] <pieterh> in germany? lol...
[Wednesday 22 September 2010] [17:25:20] <pieterh> there’s some kind of wine festival in bratislava next weekend
[Wednesday 22 September 2010] [17:25:49] <pieterh> hey, mato, random question about messages
[Wednesday 22 September 2010] [17:26:08] <pieterh> i vaguely recall El Sustrik saying that zmq_send nullifies the message it sends
[Wednesday 22 September 2010] [17:26:24] <pieterh> even insisting this should be clearly explained in the guide
[Wednesday 22 September 2010] [17:26:33] <mato> zmq_recv does
[Wednesday 22 September 2010] [17:26:39] <mato> send doesn’t afaik
[Wednesday 22 September 2010] [17:26:40] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 22 September 2010] [17:26:47] <pieterh> ? recv nullifies the message?
[Wednesday 22 September 2010] [17:26:56] * pieterh needs another glass of vino to understand this
[Wednesday 22 September 2010] [17:27:15] <pieterh> ah, before receiving, previous contents are discarded...
[Wednesday 22 September 2010] [17:28:25] <pieterh> mato: nah, it was send
[Wednesday 22 September 2010] [17:28:32] <pieterh> i found the line in the userguide:
[Wednesday 22 September 2010] [17:28:33] <pieterh> Note than when you have passed a message to zmq_send[3], 0MQ will clear the message, i.e. set the data to empty. You cannot send the same message twice, and you cannot access the message data after sending it.
[Wednesday 22 September 2010] [17:28:44] <pieterh> yet this is wrong
[Wednesday 22 September 2010] [17:28:53] Quit plol has left this server (Ping timeout: 276 seconds).
[Wednesday 22 September 2010] [17:29:15] <pieterh> could you bounce this off Sustrik?
[Wednesday 22 September 2010] [17:30:04] <mato> that’s wrong
[Wednesday 22 September 2010] [17:30:12] <mato> of course you can send a message multiple times
[Wednesday 22 September 2010] [17:30:15] <mato> i do it all the time
[Wednesday 22 September 2010] [17:30:18] <starkdg> i thought zmq_msg_close() is what kills messages ?
[Wednesday 22 September 2010] [17:30:24] <mato> yes, precisely
[Wednesday 22 September 2010] [17:30:42] <mato> recv is special in that it destroys the content of the message being received into (if any)
[Wednesday 22 September 2010] [17:30:46] <mato> which makes sense
[Wednesday 22 September 2010] [17:30:50] <mato> send does not touch it
[Wednesday 22 September 2010] [17:30:51] <pieterh> mato: that all makes sense
[Wednesday 22 September 2010] [17:31:13] <pieterh> yet I wrote that after sustrik explicitly asked me to emphasize that
[Wednesday 22 September 2010] [17:31:20] <pieterh> maybe he meant recv not send...
[Wednesday 22 September 2010] [17:31:25] <pieterh> or i noted it wrong
[Wednesday 22 September 2010] [17:31:32] <mato> probably
[Wednesday 22 September 2010] [17:31:47] <pieterh> send is 100% safe?
[Wednesday 22 September 2010] [17:31:57] <mato> it’s quite obviously wrong, i have lots of code that sends the same msg_t X times
[Wednesday 22 September 2010] [17:32:00] <pieterh> kurwa
[Wednesday 22 September 2010] [17:32:03] <pieterh> ok, thanks...
[Wednesday 22 September 2010] [17:32:04] <mato> :-)
[Wednesday 22 September 2010] [17:32:58] <starkdg> you can recieve and send the same message, right ? ive been doing that
[Wednesday 22 September 2010] [17:33:49] <mato> yes
[Wednesday 22 September 2010] [17:58:24] Quit mankins has left this server (Quit: mankins).
[Wednesday 22 September 2010] [18:01:59] Quit ModusPwnens has left this server.
*** on Wed Sep 22 18:31:08 2010
[Wednesday 22 September 2010] [18:31:08] Topic The channel topic is “Welcome!”.
[Wednesday 22 September 2010] [18:31:12] Mode Channel modes: no messages from outside, topic protection
[Wednesday 22 September 2010] [18:31:12] Created This channel was created on 2010-02-05 10:44.
[Wednesday 22 September 2010] [18:31:28] Quit travlr has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [18:47:50] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 22 September 2010] [18:50:43] Quit starkdg has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [18:58:33] Part jashmenn has left this channel.
[Wednesday 22 September 2010] [19:01:12] Nick MJW is now known as mjw9100.
[Wednesday 22 September 2010] [19:08:43] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [19:18:53] Quit starkdg has left this server (Ping timeout: 245 seconds).
[Wednesday 22 September 2010] [19:21:12] Part mjw9100 has left this channel.
[Wednesday 22 September 2010] [19:24:04] Part mjw9100 has left this channel.
[Wednesday 22 September 2010] [19:25:33] Quit friendshi has left this server (Quit: friendshi).
[Wednesday 22 September 2010] [19:42:25] Quit starkdg has left this server (Ping timeout: 252 seconds).
[Wednesday 22 September 2010] [19:42:38] Quit pieterh has left this server (Ping timeout: 245 seconds).
[Wednesday 22 September 2010] [19:54:03] Part mikejs has left this channel.
[Wednesday 22 September 2010] [20:56:45] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Wednesday 22 September 2010] [21:03:03] Quit andrewvc has left this server (Quit: andrewvc).
[Wednesday 22 September 2010] [21:59:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 22 September 2010] [22:40:39] Quit friendshi has left this server (Client Quit).
[Wednesday 22 September 2010] [22:45:48] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 22 September 2010] [23:03:12] Quit grahamalot has left this server (Quit: grahamalot).
[Wednesday 22 September 2010] [23:46:30] Quit yrashk has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [00:28:54] <Pe_Ell> anyone know if there is a roadmap for release dates? Just wondering since I didn’t notice one on github or the site off hand...
[Thursday 23 September 2010] [01:11:17] Quit BooTheHamster1 has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:11:37] Join BooTheHamster1 has joined this channel (
~boo@94.25.51.179).
[Thursday 23 September 2010] [01:24:51] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:27:37] Quit BooTheHamster1 has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [01:39:53] <guido_g> nope, there is no such thing
[Thursday 23 September 2010] [01:40:21] <guido_g> standard quote is “it’ll be released when its ready” :)
[Thursday 23 September 2010] [01:42:47] <Pe_Ell> heh, k
[Thursday 23 September 2010] [01:43:23] <Pe_Ell> there’s a feature in 2.1 that I’m waiting for... wasn’t sure how stable the git branch was... guess I’ll just keep on waiting... :)
[Thursday 23 September 2010] [01:43:37] <guido_g> if you’re around longer, ask pieterh, mato or sustrik, they do know the details
[Thursday 23 September 2010] [01:44:20] <Pe_Ell> k. I’ll just leave this open for now while I read more code/docs...
[Thursday 23 September 2010] [01:55:59] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [01:56:47] Quit plol has left this server (Ping timeout: 240 seconds).
[Thursday 23 September 2010] [01:59:54] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [02:13:01] Quit raz has left this server (Quit: fump).
[Thursday 23 September 2010] [02:58:15] Quit andrewvc has left this server (Quit: andrewvc).
[Thursday 23 September 2010] [03:19:36] Quit plol has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [03:42:56] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [04:36:10] Nick
mikko__ is now known as mikko.
[Thursday 23 September 2010] [05:00:00] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 23 September 2010] [05:25:10] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 23 September 2010] [05:49:09] Quit jugg has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [06:42:11] Join plq has joined this channel (
~plq@78.189.30.218).
[Thursday 23 September 2010] [08:25:06] Part mjw9100 has left this channel.
[Thursday 23 September 2010] [08:53:57] <pieter_hintjens> hi folk, it’s quiet today
[Thursday 23 September 2010] [08:54:21] <pieter_hintjens>
*folks
[Thursday 23 September 2010] [08:54:32] <pieter_hintjens> I’ve pushed out a new version of the user guide
[Thursday 23 September 2010] [09:01:50] Quit mytrile has left this server (Ping timeout: 252 seconds).
[Thursday 23 September 2010] [09:28:17] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [09:55:49] Quit sbahra has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [09:56:49] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [10:01:27] Quit plol has left this server (Read error: Operation timed out).
[Thursday 23 September 2010] [10:18:19] Quit plq has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [10:39:02] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Thursday 23 September 2010] [10:58:13] Quit mytrile has left this server (Quit: Leaving.).
[Thursday 23 September 2010] [11:19:34] <friendshi> hi, I’m having an issue when trying to do non-block recv... I always immediately receive an exception ‘Resource temporarily unavailable’, anyone know where i might be going wrong?
[Thursday 23 September 2010] [11:20:30] <guido_g> maybe the connect wasn’t done at this point in time
[Thursday 23 September 2010] [11:20:52] <friendshi> hmm it works fine with i do blocking, and i have tried sleeping for 5+ seconds between connect and recv
[Thursday 23 September 2010] [11:21:13] <guido_g> because the real connection handling is in a seperate thread, you might do the receive while the underlying connection is not yet established
[Thursday 23 September 2010] [11:23:33] <friendshi> i dont think that can be it... when i do it without block it begins reading my messages immediately... unless i am misunderstanding what you are suggesting
[Thursday 23 September 2010] [11:23:42] <friendshi>
*without nonblock
[Thursday 23 September 2010] [11:24:04] <guido_g> i didn’t suggest something
[Thursday 23 September 2010] [11:24:19] <friendshi> sorry i just mean, what you are saying the issue could be
[Thursday 23 September 2010] [11:24:40] <guido_g> one idea is to make the first receive blocking and all others not
[Thursday 23 September 2010] [11:25:01] <guido_g> if this works, it’s time for an issue i’d say
[Thursday 23 September 2010] [11:26:26] * guido_g checks the docu
[Thursday 23 September 2010] [11:26:27] <friendshi> same issue :(
[Thursday 23 September 2010] [11:26:45] <friendshi> it does receive first message fine, then immediate fail on nonblock recv
[Thursday 23 September 2010] [11:26:53] <guido_g> then try to make a small program that shows the problem and show the code
[Thursday 23 September 2010] [11:27:08] <friendshi> sure
[Thursday 23 September 2010] [11:45:32] Nick rbancrof1 is now known as rbancroft.
[Thursday 23 September 2010] [11:45:52] <vagn> greetings. Is there an example somewhere on how to shutdown a ZMQ_REP socket so the otherside doesn’t hang when the service returns?
[Thursday 23 September 2010] [11:46:30] Join ChicagoBud has joined this channel (
~bud@208.78.39.48).
[Thursday 23 September 2010] [11:46:57] <vagn> I want to stop getting requests, but finish all the replies, then exit the program.
[Thursday 23 September 2010] [11:47:48] <vagn> but the requests are queued, right? so the other side things it should get a reply. so basically I have to shutdown the incoming queue, too.
[Thursday 23 September 2010] [11:47:57] <vagn>
*thinks
[Thursday 23 September 2010] [11:53:19] <guido_g> i’d do this kind of synchronization with explicit messages, much easier imho
[Thursday 23 September 2010] [11:54:45] <vagn> thinking some more... that still doesn’t cure tha case where the server crashes.
[Thursday 23 September 2010] [11:56:02] <vagn> maybe I should start over and do things with pub/sub or multicast. I don’t care if I lose a message. the next one that comes along is good enough.
[Thursday 23 September 2010] [12:14:43] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 23 September 2010] [12:14:56] Quit
kayiwa_ has left this server (Quit:
kayiwa_).
[Thursday 23 September 2010] [12:16:33] <friendshi> i find that my problem was that in pyzmq document/source it states ‘If NOBLOCK is set, this method will return None if a message is not ready’ but that is untrue, I believe it is throwing exception is no message exist
[Thursday 23 September 2010] [12:16:55] <friendshi> maybe i am misreading or misunderstanding though
[Thursday 23 September 2010] [12:22:21] <keffo> oh by my might beard! I hate 3g
[Thursday 23 September 2010] [12:37:51] Quit xla has left this server (Quit: leaving).
[Thursday 23 September 2010] [12:58:56] <bgranger> friendshi: hi
[Thursday 23 September 2010] [13:19:51] Quit vagn has left this server (Quit: Ex-Chat).
[Thursday 23 September 2010] [14:56:38] Quit mytrile has left this server (Ping timeout: 272 seconds).
[Thursday 23 September 2010] [15:04:07] Quit miguelito has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [15:39:01] Notice -Martinp23- :[Global Notice] Hi folks. One of our servers (jordan) needs to undergo emergency maintenance in the next 30 minutes. The maintenance window will be 48 hours or so. The result of this will be a netsplit as 3000 users, who have already been notified seperately, disconnect. I’m sorry for the inconvenience. If you’ve any questions, pop into #freenode. Thanks!
[Thursday 23 September 2010] [15:40:44] Quit keffo has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [16:00:40] Quit miguelito has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:40] Quit yrashk has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:41] Quit mato has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:00:41] Quit jond has left this server (
*.net
*.split).
[Thursday 23 September 2010] [16:33:01] Quit plol has left this server (Ping timeout: 265 seconds).
[Thursday 23 September 2010] [16:42:38] Quit xla has left this server (Ping timeout: 264 seconds).
[Thursday 23 September 2010] [16:44:07] Quit halfie has left this server (Quit: Leaving).
[Thursday 23 September 2010] [17:01:17] Quit swapi has left this server (Quit: Page closed).
[Thursday 23 September 2010] [17:13:05] Quit bgranger has left this server (Read error: Operation timed out).
[Thursday 23 September 2010] [17:22:48] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 23 September 2010] [17:23:03] Quit miguelito has left this server (Ping timeout: 245 seconds).
[Thursday 23 September 2010] [18:01:00] Quit andrewvc has left this server (Quit: andrewvc).
[Thursday 23 September 2010] [18:15:57] Quit friendshi has left this server (Remote host closed the connection).
[Thursday 23 September 2010] [19:35:43] Part vanadium has left this channel.
[Thursday 23 September 2010] [19:37:44] Quit
mw_ has left this server (Quit:
mw_).
[Thursday 23 September 2010] [19:39:25] Quit larrytheliquid has left this server (Quit: larrytheliquid).
[Thursday 23 September 2010] [19:47:22] Quit
mw_ has left this server (Quit:
mw_).
[Thursday 23 September 2010] [20:43:27] Quit
miguelito_ has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [20:49:45] Quit iFire has left this server (Read error: Connection reset by peer).
[Thursday 23 September 2010] [20:55:26] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [21:21:38] <jhawk28> does any other protocols than tcp work on Windows?
[Thursday 23 September 2010] [21:50:58] Quit xla has left this server (Ping timeout: 245 seconds).
[Thursday 23 September 2010] [22:28:06] Quit bgranger has left this server (Quit: bgranger).
[Thursday 23 September 2010] [22:31:23] Quit jhawk28 has left this server (Ping timeout: 276 seconds).
[Thursday 23 September 2010] [22:32:57] Part starkdg has left this channel.
[Thursday 23 September 2010] [23:55:51] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [00:00:08] Quit mjw9100 has left this server (Remote host closed the connection).
[Friday 24 September 2010] [01:38:56] Quit jsimmons has left this server (Quit: Leaving).
[Friday 24 September 2010] [02:24:03] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [02:27:48] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [02:28:13] Quit guido_g has left this server (Quit: Boom...).
[Friday 24 September 2010] [02:33:22] Quit pieter_hintjens has left this server (Ping timeout: 255 seconds).
[Friday 24 September 2010] [03:09:06] Nick Nickname already in use. Trying
travlr__.
[Friday 24 September 2010] [03:09:11] Topic The channel topic is “Welcome!”.
[Friday 24 September 2010] [03:09:12] CTCP Received Version request from frigg.
[Friday 24 September 2010] [03:09:12] Notice -NickServ- travlr__ is not a registered nickname.
[Friday 24 September 2010] [03:09:15] Mode Channel modes: no messages from outside, topic protection
[Friday 24 September 2010] [03:09:15] Created This channel was created on 2010-02-05 10:44.
[Friday 24 September 2010] [03:09:43] Quit
travlr_ has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [03:10:14] Quit Pe_Ell has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [03:19:13] Quit SecretAgent has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [03:20:26] Quit AlexB has left this server (Ping timeout: 264 seconds).
[Friday 24 September 2010] [03:20:40] Join AlexB has joined this channel (
alex@brasetvik.com).
[Friday 24 September 2010] [03:33:43] Quit Pe-Ell has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [03:34:50] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [04:08:28] Quit xla has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [04:59:18] Quit drbobbeaty has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [05:02:02] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [05:07:43] Join xla has joined this channel (
~xla@89.246.67.229).
[Friday 24 September 2010] [05:25:26] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 24 September 2010] [05:30:46] Quit guido_g has left this server (Quit: Boom...).
[Friday 24 September 2010] [05:34:07] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [07:27:21] <Zao> Assuming you manage to build it, I can’t see why OpenPGM wouldn’t work.
[Friday 24 September 2010] [07:28:31] Quit keffo has left this server (Quit: Leaving).
[Friday 24 September 2010] [07:32:40] Part
i__ has left this channel.
[Friday 24 September 2010] [07:40:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Friday 24 September 2010] [08:48:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [08:55:50] Quit plol has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [09:07:09] Part mjw9100 has left this channel.
[Friday 24 September 2010] [09:29:53] Part mjw9100 has left this channel.
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r7073557 10/ src/org/zeromq/ZMQ.java : Javadoc, reduction of the visibility of a lot of flags reduced to private because they are only used inside public setters/getters. Fix of some checkstyle warnings. -
http://bit.ly/aGlNur[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r34d656f 10/ src/org/zeromq/ZMQ.java :
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Deprecation of ZMQ.UPSTREAM and ZMQ.DOWNSTREAM
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Socket.subscribe and Socket.unsubscribe only on a Socket of Type ZMQ.SUB -
http://bit.ly/cQRpmM[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: 03Stefan Majer 07master * r86512f8 10/ src/org/zeromq/ZMQ.java :
[Friday 24 September 2010] [09:45:33] <CIA-20> jzmq: Revert the subscribe/unsubscribe check,
[Friday 24 September 2010] [09:45:34] <CIA-20> jzmq: Whitespace and identation with zeromq rules -
http://bit.ly/9wrDyQ[Friday 24 September 2010] [09:45:35] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * r3ba6cc1 10/ src/org/zeromq/ZMQ.java : Pulled changes from Stefan Majer to improve documentation and visibility. Made minor modifications. -
http://bit.ly/awIGSC[Friday 24 September 2010] [09:55:50] Part mjw9100 has left this channel.
[Friday 24 September 2010] [10:53:29] <codebeaker> hey, quick sanity check - can I (should I?) include the zeromq repository in my own project as a git submodule ?
[Friday 24 September 2010] [11:05:00] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [11:10:43] Quit rhi has left this server (Client Quit).
[Friday 24 September 2010] [11:15:43] Quit bgranger has left this server (Quit: bgranger).
[Friday 24 September 2010] [11:16:58] <cremes> codebeaker: that’s a good question for some git experts... i don’ t think there are any in this channel
[Friday 24 September 2010] [11:18:04] <codebeaker> cremes: I don’t need a git expert, my question was more about whether it makes sense, if you distribute a closed source application, between a couple of devlopers (mac and windows targets) how are other people typically including the mq sources in their projects
[Friday 24 September 2010] [11:18:46] <codebeaker> I noticed there were separate downloads available for Windows and Unix, so that would lead me to believe you have to build them differently
[Friday 24 September 2010] [11:20:51] <cremes> i don’t know how to answer your question... hopefully someone else will come along who can
[Friday 24 September 2010] [11:21:09] <codebeaker> thanks cremes - appreciate the attention anyway
[Friday 24 September 2010] [11:21:33] <codebeaker> basically, I could include the whole repository from mq in my own, and then specify build flags - but I’m not sure how smart that would be
[Friday 24 September 2010] [11:31:29] <pieter_hintjens> codebeaker: including 0MQ as a git submodule would be a pretty good solution, yes
[Friday 24 September 2010] [11:32:09] <pieter_hintjens> at least until 0MQ becomes a standard package on the various distros
[Friday 24 September 2010] [11:32:29] <pieter_hintjens> and definitely a good idea if you want specific versions or branches of 0MQ
[Friday 24 September 2010] [11:33:08] <pieter_hintjens> but the distribution packages are identical except for line endings and compression format
[Friday 24 September 2010] [11:35:49] <codebeaker> pieter_hintjens: ^ thanks that’s exactly the info I needed :-D
[Friday 24 September 2010] [11:35:58] <codebeaker> bew, mq is freakin awesome
[Friday 24 September 2010] [11:36:01] <codebeaker> btw*
[Friday 24 September 2010] [11:36:02] Quit codebeaker has left this server (Quit: codebeaker).
[Friday 24 September 2010] [11:36:04] <pieter_hintjens> codebeaker: hopefully it’s also accurate :-)
[Friday 24 September 2010] [12:00:13] Quit plol has left this server (Ping timeout: 252 seconds).
[Friday 24 September 2010] [12:12:10] <mjw9100> folks, just confirming that an “ordinary” fd for zmq_poll() can be an epoll fd?
[Friday 24 September 2010] [12:19:23] <mjw9100> i.e. can I set zmq_pollitem_t.fd to an epoll() fd?
[Friday 24 September 2010] [12:19:56] <mjw9100> according to the epoll() Q&A, these epoll fd’s are selectable, etc.
[Friday 24 September 2010] [12:23:19] Quit jhawk28 has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [12:24:19] <jhawk28> hello
[Friday 24 September 2010] [12:24:59] <jhawk28> having a problem moving the compiled version of the windows libraries to another machine
[Friday 24 September 2010] [12:55:47] <praveenbm5> hello all
[Friday 24 September 2010] [12:56:03] <praveenbm5> i hope i am in the right room to discuss ZeroMQ
[Friday 24 September 2010] [12:56:12] Quit omarkj has left this server (Quit: omarkj).
[Friday 24 September 2010] [12:56:42] <codebeaker> praveenbm5: the name says it all!
[Friday 24 September 2010] [12:56:46] <praveenbm5> is anybody alive here
[Friday 24 September 2010] [12:57:15] <praveenbm5> i was looking on the website regarding some guidance using zmq_poll
[Friday 24 September 2010] [12:57:53] <praveenbm5> i just posted a comment there
[Friday 24 September 2010] [13:03:35] <codebeaker> ^ I’m inqo mq for about 5 minutes now, what’s the behaviour if you broadcast a message to an upstream (is the terminology the right way round there ) - but nothing is listening there yet, is it lost ?
[Friday 24 September 2010] [13:04:58] <praveenbm5> its not lost
[Friday 24 September 2010] [13:05:25] <praveenbm5> queues exist even when the end point doesnt yet exist
[Friday 24 September 2010] [13:05:53] <praveenbm5> i suggest you read the guide on zeromq.org
[Friday 24 September 2010] [13:05:53] <codebeaker> ah, ok - great - i’m actually looking into it for writing a plugin for winamp that fires off a list of whats playing
[Friday 24 September 2010] [13:06:14] <codebeaker> (and I can’t be sure the other app is running)
[Friday 24 September 2010] [13:06:52] <praveenbm5> sorry i got your questiong wrong
[Friday 24 September 2010] [13:07:18] <praveenbm5> you can connect to an end point from a zmq socket before the endpoint exists
[Friday 24 September 2010] [13:07:19] <jhawk28> codebeaker: the new terminology for “upstream” is “push”
[Friday 24 September 2010] [13:07:32] <praveenbm5> but regarding messages
[Friday 24 September 2010] [13:07:38] <codebeaker> no problem, winamp loads .dlls like plugins, and I wanted that plugin to shout out (over zmq) “hey, something played in winamp” - what happens if something isn’t listening ?
[Friday 24 September 2010] [13:08:07] <praveenbm5> push is for parallel worker based execution
[Friday 24 September 2010] [13:08:14] <jhawk28> it sits in the local queue till the socket can connect I believe
[Friday 24 September 2010] [13:08:49] <jhawk28> upstream/downstream == push/pull
[Friday 24 September 2010] [13:09:31] <jhawk28> the only one that doesn’t “keep” the message is the pub socket when noone has connected (I believe)
[Friday 24 September 2010] [13:10:15] <codebeaker> jhawk28: great, and that local queue is transient, in my example - if the closed winamp, and the plugin unloaded - it would be lost, right ?
[Friday 24 September 2010] [13:12:04] <jhawk28> yes, I think so
[Friday 24 September 2010] [13:12:29] <codebeaker> jhawk28: thanks, I’m going to screw around in the Ruby console a little - but that sounds like exactly the behaviour I was looking for
[Friday 24 September 2010] [13:12:41] <jhawk28> you would have to manually persist if you dont want it transient
[Friday 24 September 2010] [13:13:05] <codebeaker> yeah, jhawk28 - I’m looking for it not to persist, or at least to know if it was read off (I suppose I could always examine the local queue)
[Friday 24 September 2010] [13:13:06] <jhawk28> easiest way to see is to download one of the samples and then compile it
[Friday 24 September 2010] [13:13:22] <jhawk28> start them both a couple of times, but in different orders
[Friday 24 September 2010] [13:13:27] <codebeaker> also, lsat question (and I can infer a litlte from the guide) the ipc:// dsn specification, it is portable across windows/unix, I assume ?
[Friday 24 September 2010] [13:13:43] <jhawk28> I havent gotten ipc to work on my windows machine
[Friday 24 September 2010] [13:13:48] <jhawk28> the inproc seems to
[Friday 24 September 2010] [13:14:25] <codebeaker> ahh - I was sortof counting on that - however since it’s trivial, and it’s not really IPC, if it’s two separate apps a .dll in winamp, and something else standalone - maybe IPC is the wrong paradigm now
[Friday 24 September 2010] [13:14:35] <jhawk28> from the man page “The inter-process transport passes messages between local processes using a system-dependent IPC mechanism.”
[Friday 24 September 2010] [13:15:12] <jhawk28> vs “The in-process transport passes messages via memory directly between threads sharing a single MQ context”
[Friday 24 September 2010] [13:15:51] <codebeaker> ah, so IPC is certainly somehow better suited, but a socket would work just as well
[Friday 24 September 2010] [13:18:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [13:19:03] <jhawk28> they are both “sockets”, but the IPC is dependent on the OS
[Friday 24 September 2010] [13:19:08] <codebeaker> ah, ok
[Friday 24 September 2010] [13:19:20] <jhawk28> so if you want cross platform, you probably need inproc
[Friday 24 September 2010] [13:19:30] <guido_g> no
[Friday 24 September 2010] [13:19:37] <codebeaker> inproc wouldn’t work for talking from a winamp process to another process all together
[Friday 24 September 2010] [13:19:44] <guido_g> ipc is between different processes
[Friday 24 September 2010] [13:20:02] <guido_g> on windows, where ipc is not available, use
tcp://127.0.0.1[Friday 24 September 2010] [13:20:03] <codebeaker> I assume I need to go out-of-band if IPC doesn’t work on Windows, and use a real socket, like TCP or sth. that windows supports
[Friday 24 September 2010] [13:20:14] Quit titto has left this server (Quit: Ex-Chat).
[Friday 24 September 2010] [13:20:16] <guido_g> ipc is a real socket
[Friday 24 September 2010] [13:20:26] <codebeaker> ah, but a unix file socket, right ?
[Friday 24 September 2010] [13:20:31] <guido_g> right
[Friday 24 September 2010] [13:20:47] <codebeaker> ok, I assumed (foolishly) that ipc:// on windows would fall-back to tcp://
[Friday 24 September 2010] [13:22:41] <jhawk28> so, inproc is between threads, but not between processes
[Friday 24 September 2010] [13:22:45] <jhawk28> ?
[Friday 24 September 2010] [13:22:52] <codebeaker> jhawk28: certainly sounds like that
[Friday 24 September 2010] [13:24:35] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [13:27:44] <jhawk28> has anyone done the zmq and jzmq build on vs2010 on Win7_64bit?
[Friday 24 September 2010] [13:28:34] <guido_g> have a look at the mailing list, it’s been discussed there afair
[Friday 24 September 2010] [13:38:01] Quit keffo has left this server (Ping timeout: 255 seconds).
[Friday 24 September 2010] [13:41:27] Part mjw9100 has left this channel.
[Friday 24 September 2010] [13:48:02] Quit keffo has left this server (Ping timeout: 264 seconds).
[Friday 24 September 2010] [13:53:11] <jond> codebreaker: ipc isnt implemented on windows, inproc is between threads.
[Friday 24 September 2010] [13:53:20] Part praveenbm5 has left this channel.
[Friday 24 September 2010] [14:01:12] <codebeaker> jond: rgr that (from earlier!) - pretty sure i’ll go with a tcp socket on both platforms
[Friday 24 September 2010] [14:01:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:14:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:23:18] Quit keffo has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [14:33:26] Quit mytrile has left this server (Quit: Leaving.).
[Friday 24 September 2010] [14:38:47] <jhawk28> hurrah! I got VS2010 to compile zmq and jzmq
[Friday 24 September 2010] [14:38:55] <codebeaker> w00t, well done :-D
[Friday 24 September 2010] [14:48:17] Quit mytrile has left this server (Read error: Operation timed out).
[Friday 24 September 2010] [14:53:02] Quit sustrik has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [15:01:35] Quit xla has left this server (Quit: leaving).
[Friday 24 September 2010] [15:18:40] <rgl> hi
[Friday 24 September 2010] [15:23:09] <codebeaker> hi rgl
[Friday 24 September 2010] [15:33:27] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [15:45:33] Quit starkdg has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [15:47:10] Part mjw9100 has left this channel.
[Friday 24 September 2010] [16:08:53] Quit
mw_ has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [16:18:51] <jond> codebeaker: ipc is faster on linux, than tcp loopback; on solaris it made no difference as solaris loopback is optimised
[Friday 24 September 2010] [16:19:29] <jond> sustrik: how did the presentation go?
[Friday 24 September 2010] [16:55:58] Quit jhawk28 has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [16:56:36] Quit keffo has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [17:06:00] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [17:25:24] Quit codebeaker has left this server (Quit: codebeaker).
[Friday 24 September 2010] [17:26:15] Quit codebeaker has left this server (Client Quit).
[Friday 24 September 2010] [17:27:25] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Friday 24 September 2010] [17:34:07] Quit jhawk28 has left this server (Remote host closed the connection).
[Friday 24 September 2010] [17:40:26] Quit mytrile has left this server (Read error: Connection reset by peer).
[Friday 24 September 2010] [17:48:18] Quit ysynopsis has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [18:24:15] Quit rgl has left this server (Ping timeout: 265 seconds).
[Friday 24 September 2010] [18:44:37] Nick _rgl is now known as rgl.
[Friday 24 September 2010] [19:00:43] Quit bennymack has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [19:19:18] Quit mytrile has left this server (Ping timeout: 245 seconds).
[Friday 24 September 2010] [19:20:18] Quit rgl has left this server (Ping timeout: 240 seconds).
[Friday 24 September 2010] [19:20:40] Quit friendshi has left this server (Remote host closed the connection).
[Friday 24 September 2010] [20:47:31] Quit starkdg has left this server (Quit: goodbye).
[Friday 24 September 2010] [21:56:52] Quit iFire has left this server (Read error: Connection reset by peer).
*** on Fri Sep 24 22:16:23 2010
[Friday 24 September 2010] [22:16:23] Topic The channel topic is “Welcome!”.
[Friday 24 September 2010] [22:16:27] Mode Channel modes: no messages from outside, topic protection
[Friday 24 September 2010] [22:16:27] Created This channel was created on 2010-02-05 10:44.
[Friday 24 September 2010] [22:30:23] Quit
mw_ has left this server (Quit:
mw_).
[Friday 24 September 2010] [22:31:40] Quit grahamalot has left this server (Quit: grahamalot).
[Friday 24 September 2010] [23:09:34] Quit jsimmons has left this server (Ping timeout: 272 seconds).
[Friday 24 September 2010] [23:10:42] Quit jhawk28 has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [00:05:07] Quit RobertLJ has left this server (Ping timeout: 252 seconds).
[Saturday 25 September 2010] [01:20:48] Quit rgl has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [01:52:35] Quit lluad has left this server (Quit: lluad).
[Saturday 25 September 2010] [02:32:42] <sustrik> Pe_Ell: 2.1 looks more or less stable already. Please, do use it. If any bug is found, let me know.
[Saturday 25 September 2010] [02:36:19] Quit bgranger has left this server (Quit: bgranger).
[Saturday 25 September 2010] [02:50:20] <sustrik> jond: it went well, i’ll post my paper shortly
[Saturday 25 September 2010] [02:50:39] <sustrik> met a lot of linux kernel folks there
[Saturday 25 September 2010] [02:58:35] <Pe_Ell> wow... chapter3 just doubled in size..
[Saturday 25 September 2010] [02:58:51] Quit rgl has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [03:09:18] Quit keffo has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [03:25:07] <Pe_Ell> sustrik, cool. Mostly was interested in accessing the “socket” more directly so there could be some attempt of creting somelike like a Perl AnyEvent wrapper...
[Saturday 25 September 2010] [03:26:37] <Pe_Ell> wow... my english sucks...
[Saturday 25 September 2010] [03:29:47] <sustrik> Pe_Ell: yes, feel free to play with that
[Saturday 25 September 2010] [03:30:09] <sustrik> there’s ZMQ_FD socket option that allows you to do that
[Saturday 25 September 2010] [03:30:33] <Pe_Ell> yeah I was looking over the code and currently reading the guide...
[Saturday 25 September 2010] [03:30:40] <Pe_Ell> pretty entertaining and useful
[Saturday 25 September 2010] [03:30:44] <sustrik> however, it’s a bit more complex than using standard 0mq API
[Saturday 25 September 2010] [03:31:03] <sustrik> you have to keep in mind that:
[Saturday 25 September 2010] [03:31:34] <sustrik> 1. ZMQ_FD can signal even if there’s no message available (on internal 0MQ events such as “connection established”)
[Saturday 25 September 2010] [03:31:47] <sustrik> 2. ZMQ_FD is edge triggered
[Saturday 25 September 2010] [03:32:28] <sustrik> if you don’t care about that, it should work just fine
[Saturday 25 September 2010] [03:32:51] <Pe_Ell> yeah I figured I’d finish the guide before digging into the 2.1 code more...
[Saturday 25 September 2010] [04:09:07] Quit mytrile has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [05:25:29] <pieter_hintjens> Pe_Ell: I moved a lot of Ch3 into Ch2 as well
[Saturday 25 September 2010] [05:41:48] Quit mytrile has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [06:34:27] Quit mytrile has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [07:49:04] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [08:12:25] Quit jsimmons has left this server (Ping timeout: 252 seconds).
[Saturday 25 September 2010] [08:13:18] Quit rossij has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [08:34:48] Quit ysynopsis has left this server (Quit: Leaving.).
[Saturday 25 September 2010] [08:58:48] Quit keffo has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [09:33:13] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [09:54:24] Quit
fmu_ has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [09:57:04] Quit ysynopsis has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [10:41:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Saturday 25 September 2010] [11:46:14] Quit lluad has left this server (Ping timeout: 264 seconds).
[Saturday 25 September 2010] [11:46:14] Nick
lluad_ is now known as lluad.
*** on Sat Sep 25 13:29:53 2010
[Saturday 25 September 2010] [13:29:53] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [13:29:57] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [13:29:57] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [13:43:50] <Pe_Ell> pieter_hintjens: so I should start over? :) I noticed that it had shifted some, I was going to go look at the earlier chapters to see if I’d missed anything... still got a lot of code to read... :)
[Saturday 25 September 2010] [13:44:26] <pieter_hintjens> Pe_Ell: well, most of the new stuff is at the end of Ch2 and then Ch3
[Saturday 25 September 2010] [13:47:05] <Pe_Ell> yeah I figure I’d like a pretty generic AnyEvent wrapper if it’s reasonable to build... I’m still working out the application design for my project though... I just got distracted with new and shiny.... :D
[Saturday 25 September 2010] [13:57:51] <Pe_Ell> so to make sure I’m not retarded... for pubsub I can make one ZMQ_SUB socket and then just set multiple ZMQ_SUBSCRIPT options on it...
[Saturday 25 September 2010] [13:58:24] <Pe_Ell> err, ZMQ_SUBSCRIBE
[Saturday 25 September 2010] [13:58:45] <Pe_Ell> man I don’t even have a spell checker... not sure why I keep writing out the wrong words...
[Saturday 25 September 2010] [13:59:50] <Pe_Ell> ok man page confirmed what I was thinking... and looks like the string is a prefix so that answered my next question about wildcarding...
[Saturday 25 September 2010] [14:36:32] Quit pieter_hintjens has left this server (Ping timeout: 276 seconds).
[Saturday 25 September 2010] [14:55:25] <lluad> Anyone have any pointers for embedding 0mq into another systems event loop - I’m looking at using it with Qt4.
[Saturday 25 September 2010] [15:13:53] Quit lluad has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [15:13:53] Nick
lluad_ is now known as lluad.
[Saturday 25 September 2010] [15:45:01] <cremes> lluad: take a look at the 2.1 branch specifically ZM_FD and ZM_EVENTS
[Saturday 25 September 2010] [15:45:10] <lluad> Excellent, thanks.
*** on Sat Sep 25 17:20:20 2010
[Saturday 25 September 2010] [17:20:20] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [17:20:24] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [17:20:24] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [17:26:46] Quit
antares_ has left this server (Quit:
antares_).
[Saturday 25 September 2010] [17:45:16] Quit
antares_ has left this server (Quit:
antares_).
[Saturday 25 September 2010] [18:09:36] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Saturday 25 September 2010] [18:27:34] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
*** on Sat Sep 25 21:11:14 2010
[Saturday 25 September 2010] [21:11:14] Topic The channel topic is “Welcome!”.
[Saturday 25 September 2010] [21:11:18] Mode Channel modes: no messages from outside, topic protection
[Saturday 25 September 2010] [21:11:18] Created This channel was created on 2010-02-05 10:44.
[Saturday 25 September 2010] [21:48:28] Quit sbahra has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [21:53:53] Quit pasha_bastov has left this server (Ping timeout: 245 seconds).
[Saturday 25 September 2010] [22:33:30] Quit lluad has left this server (Quit: lluad).
[Saturday 25 September 2010] [22:59:26] Quit
pasha_bastov_ has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [23:15:34] Join jdroid- has joined this channel (
~user@74.72.57.182).
[Saturday 25 September 2010] [23:16:25] <jdroid-> hello. i’m reading whitepapers:design-v06. am i right that this is mapped to zeromq v 0.6, making it also quite old?
[Saturday 25 September 2010] [23:27:05] Quit iFire has left this server (Read error: Connection reset by peer).
[Saturday 25 September 2010] [23:37:37] Quit jdroid- has left this server (Remote host closed the connection).
[Saturday 25 September 2010] [23:41:07] Quit bgranger has left this server (Quit: bgranger).
[Saturday 25 September 2010] [23:43:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [01:05:48] Quit jsimmons has left this server (Quit: Leaving).
[Sunday 26 September 2010] [01:49:04] Join sbahra has joined this channel (
~sbahra@pool-74*** on Sun Sep 26 12:46:34 2010
[Sunday 26 September 2010] [12:46:34] Topic The channel topic is “Welcome!”.
[Sunday 26 September 2010] [12:46:38] Mode Channel modes: no messages from outside, topic protection
[Sunday 26 September 2010] [12:46:38] Created This channel was created on 2010-02-05 10:44.
[Sunday 26 September 2010] [13:20:48] Quit bennymack has left this server (Ping timeout: 240 seconds).
[Sunday 26 September 2010] [14:50:48] Quit xla has left this server (Ping timeout: 265 seconds).
[Sunday 26 September 2010] [15:10:08] Quit eintr has left this server (Quit: Leaving.).
[Sunday 26 September 2010] [15:16:52] Quit pasha_bastov has left this server (Quit: Leaving).
[Sunday 26 September 2010] [15:43:34] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * rbe79a9f 10/ (4 files): minor beautifications -
http://bit.ly/cH9kWR[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * rcf815e8 10/ (18 files): new interfaces for timers; the implementation is not changed yet -
http://bit.ly/buVKBu[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r8d7bf66 10/ (13 files): common base for all pollers created; the only thing it handles at the moment is ‘load’ -
http://bit.ly/cqLLHi[Sunday 26 September 2010] [15:49:22] <CIA-20> zeromq2: 03Martin Sustrik 07master * r238640a 10/ (15 files): timers properly implemented -
http://bit.ly/9clGnp[Sunday 26 September 2010] [15:54:10] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Sunday 26 September 2010] [16:19:47] Quit keffo has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [17:48:57] Quit GeekGod has left this server (Quit: GeekGod).
[Sunday 26 September 2010] [19:18:20] Quit lotrpy has left this server.
[Sunday 26 September 2010] [19:44:00] <jhawk28> hello, anyone here with windows build experience?
[Sunday 26 September 2010] [19:44:00] Quit zmqircd has left this server (Remote host closed the connection).
[Sunday 26 September 2010] [19:44:35] <jhawk28> I’m having trouble getting the windows code to run on other systems that don’t have VS2008 installed
[Sunday 26 September 2010] [20:05:07] <Zao> jhawk28: Several solutions.
[Sunday 26 September 2010] [20:05:49] <Zao> jhawk28: 1) deploy the runtime via installer. 2) deploy the runtime privately in a subdirectory of your application. 3) build your project against the static runtime, with all the downsides and pitfalls that has.
[Sunday 26 September 2010] [20:06:23] <Zao> Deploying privately with 2008 amounts to grabbing the Microsoft.VC90.CRT directory and putting it next to your executable.
[Sunday 26 September 2010] [20:06:36] <Zao> Deploying via installer can be done either via the standalone installer or via merge modules.
[Sunday 26 September 2010] [20:06:47] <jhawk28> I don’t mind installing the vs2008 redist
[Sunday 26 September 2010] [20:06:51] <Zao> Building against static runtime places severe restrictions to multi-module programs.
[Sunday 26 September 2010] [20:07:11] <Zao> Then install the runtime(s) that your application and your dependencies are built against.
[Sunday 26 September 2010] [20:07:19] <jhawk28> but, I can’t figure out how to have zmq compile against the latest version
[Sunday 26 September 2010] [20:07:19] <Zao> If you’re doing things right, that should be a single runtime version.
[Sunday 26 September 2010] [20:08:47] <jhawk28> btw, this is for the java bindings and the zmq library
[Sunday 26 September 2010] [20:10:19] <jhawk28> but it didn’t change the values in the manifest
[Sunday 26 September 2010] [20:10:46] <Zao> I feel sorry for you. Building against and deploying the right 2008 runtime is great fun.
[Sunday 26 September 2010] [20:10:50] <Zao> Especially with the hotfixes.
[Sunday 26 September 2010] [20:11:57] <jhawk28> :)
[Sunday 26 September 2010] [20:12:01] <jhawk28> :(
[Sunday 26 September 2010] [20:14:32] <jhawk28> new to the whole Visual Studio world so its mostly painful right now
[Sunday 26 September 2010] [20:14:56] <jhawk28> mostly happy that ZMQ compiles out of the box
[Sunday 26 September 2010] [20:16:30] <jhawk28> seems to cover your 3 options
[Sunday 26 September 2010] [20:19:09] Quit xla has left this server (Ping timeout: 265 seconds).
[Sunday 26 September 2010] [20:20:49] <jhawk28> ha, the statically linking only works VS2003 and before based on the comments
[Sunday 26 September 2010] [20:21:29] <jhawk28> and the including the dlls in a sub directory doesn’t work with VS2008-SP1
[Sunday 26 September 2010] [20:23:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [20:43:38] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [20:59:15] Quit bgranger has left this server (Quit: bgranger).
[Sunday 26 September 2010] [21:03:19] Part xrfang has left this channel.
[Sunday 26 September 2010] [22:14:19] Part larrytheliquid has left this channel.
[Sunday 26 September 2010] [22:18:28] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Sunday 26 September 2010] [22:25:28] Quit jhawk28 has left this server (Remote host closed the connection).
[Sunday 26 September 2010] [22:52:02] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [23:12:53] Quit iFire has left this server (Read error: Connection reset by peer).
[Sunday 26 September 2010] [23:58:44] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [00:15:22] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [00:21:13] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [00:45:31] Quit mcxx has left this server (Remote host closed the connection).
[Monday 27 September 2010] [01:10:15] Quit pasha_bastov has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [01:11:47] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:21:10] Quit pasha_bastov has left this server (Quit: Leaving).
[Monday 27 September 2010] [01:24:22] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:30:09] Quit guido_g has left this server (Quit: Boom...).
[Monday 27 September 2010] [01:32:49] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:34:01] Quit pasha_bastov has left this server (Ping timeout: 252 seconds).
[Monday 27 September 2010] [01:34:59] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [01:44:17] Part pieter_hintjens has left this channel.
[Monday 27 September 2010] [01:48:26] Quit zmqircd has left this server (Remote host closed the connection).
[Monday 27 September 2010] [01:52:33] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:53:29] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [01:53:56] Quit pasha_bastov has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [02:22:49] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [02:39:17] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [03:24:19] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [03:47:37] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [03:56:51] <CIA-20> zeromq2: 03Martin Sustrik 07master * r30c1cba 10/ src/poller_base.cpp : latency optimisation for the case where there are no timers -
http://bit.ly/d8hem9[Monday 27 September 2010] [03:56:51] <CIA-20> zeromq2: 03Martin Sustrik 07master * re2f834d 10/ doc/zmq_getsockopt.txt : ZMQ_FD and ZMQ_EVENTS documentation added -
http://bit.ly/9nWC7q[Monday 27 September 2010] [04:00:08] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [04:32:56] Quit sustrik has left this server (Quit: Leaving).
[Monday 27 September 2010] [04:55:22] Part pasha_bastov has left this channel (“Leaving”).
[Monday 27 September 2010] [04:55:26] Quit plol has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [04:56:52] Quit
pasha_bastov_ has left this server (Client Quit).
[Monday 27 September 2010] [04:57:14] Join pasha_bastov has joined this channel (
~Pasha@89.237.245.22).
[Monday 27 September 2010] [05:03:57] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 27 September 2010] [05:17:46] <CIA-20> zeromq2: 03Martin Sustrik 07master * r16c3884 10/ (4 files in 2 dirs): MSVC build fixed -
http://bit.ly/abDASN[Monday 27 September 2010] [05:25:07] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 27 September 2010] [06:05:58] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:19:58] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:26:26] Quit jond has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [06:38:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:40:20] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:42:02] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:47:48] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:50:37] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:52:19] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [06:55:18] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [07:04:55] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [07:36:09] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [08:11:06] Quit pasha_bastov has left this server (Quit: Leaving).
[Monday 27 September 2010] [08:30:02] Quit plol has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [08:49:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [09:29:25] Quit yawn has left this server (Quit: yawn).
[Monday 27 September 2010] [09:43:47] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [09:44:44] Quit sbahra has left this server (Remote host closed the connection).
[Monday 27 September 2010] [09:47:28] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [09:57:08] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [10:08:45] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [10:28:41] Quit guido_g has left this server (Quit: Boom...).
[Monday 27 September 2010] [10:34:29] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [10:54:21] Quit mytrile has left this server (Quit: Leaving.).
[Monday 27 September 2010] [11:00:08] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [11:12:10] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:27:38] Quit xla has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [11:28:56] Quit iFire has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:29:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Monday 27 September 2010] [11:52:56] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [11:55:02] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [11:56:03] <shawn42> hello
[Monday 27 September 2010] [11:56:03] Quit zmqircd has left this server (Remote host closed the connection).
[Monday 27 September 2010] [11:57:05] <shawn42> has anyone here done any P2P stuff w/ zeromq?
[Monday 27 September 2010] [12:04:07] Quit drbobbeaty has left this server (Read error: Connection reset by peer).
[Monday 27 September 2010] [12:04:07] Nick
drbobbeaty_ is now known as drbobbeaty.
[Monday 27 September 2010] [12:54:26] Quit shawn42 has left this server (Quit: shawn42).
[Monday 27 September 2010] [12:58:31] Quit
antares_ has left this server (Quit:
antares_).
[Monday 27 September 2010] [13:30:28] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [13:57:28] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [14:33:08] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [14:55:58] Quit omarkj has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [15:53:23] Quit keffo has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [16:16:41] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Monday 27 September 2010] [16:24:35] Quit plol has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [16:33:06] Quit
antares_ has left this server (Quit:
antares_).
[Monday 27 September 2010] [16:44:48] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [16:46:41] Quit ModusPwnens has left this server (Ping timeout: 276 seconds).
[Monday 27 September 2010] [17:02:57] Quit keffo has left this server (Read error: Operation timed out).
[Monday 27 September 2010] [17:05:39] <kristianpaul> Hello
[Monday 27 September 2010] [17:06:48] <kristianpaul> :/
[Monday 27 September 2010] [17:29:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Monday 27 September 2010] [17:29:39] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Monday 27 September 2010] [17:32:49] Quit zmqircd has left this server (Remote host closed the connection).
*** on Mon Sep 27 17:35:20 2010
[Monday 27 September 2010] [17:35:20] Topic The channel topic is “Welcome!”.
[Monday 27 September 2010] [17:35:24] Mode Channel modes: no messages from outside, topic protection
[Monday 27 September 2010] [17:35:24] Created This channel was created on 2010-02-05 10:44.
[Monday 27 September 2010] [17:37:07] Quit keffo has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [17:38:32] Join jond has joined this channel (
~jon@87.74.16.24).
[Monday 27 September 2010] [17:40:13] Quit ModusPwnens has left this server.
[Monday 27 September 2010] [17:50:58] Quit andrewvc has left this server (Ping timeout: 272 seconds).
[Monday 27 September 2010] [18:02:51] Quit bgranger has left this server (Quit: bgranger).
[Monday 27 September 2010] [18:08:21] <ipc> howdy; does anyone know if you can run zmq ‘threadless’; i.e. one process use a TCP-bound zmq socket?
[Monday 27 September 2010] [18:08:53] <ipc> the context docs suggest you shouldn’t init with zero threads unless you’re only using in-process sockets
[Monday 27 September 2010] [18:15:19] Part ipc has left this channel.
[Monday 27 September 2010] [18:33:51] Quit bgranger has left this server (Read error: Operation timed out).
[Monday 27 September 2010] [18:45:19] <kristianpaul> ahh solved
[Monday 27 September 2010] [18:45:24] <kristianpaul> i missed the lib
[Monday 27 September 2010] [18:45:28] <kristianpaul> bye
[Monday 27 September 2010] [18:45:30] Part kristianpaul has left this channel.
[Monday 27 September 2010] [18:57:58] <jhawk28> hello
[Monday 27 September 2010] [19:24:42] Quit aleator has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:24:43] Quit neopallium has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:24:43] Quit Zao has left this server (
*.net
*.split).
[Monday 27 September 2010] [19:53:07] Quit xla has left this server (Ping timeout: 240 seconds).
[Monday 27 September 2010] [20:21:23] Quit andrewvc has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [20:35:53] Quit xla has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [20:51:21] Quit xla has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [20:57:04] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Monday 27 September 2010] [21:16:00] Quit keffo has left this server (Ping timeout: 265 seconds).
[Monday 27 September 2010] [21:45:51] Quit jhawk28 has left this server (Remote host closed the connection).
[Monday 27 September 2010] [22:41:48] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Monday 27 September 2010] [23:39:45] Quit
mw_ has left this server (Quit:
mw_).
[Tuesday 28 September 2010] [00:00:16] Quit grahamalot has left this server (Quit: grahamalot).
[Tuesday 28 September 2010] [01:00:03] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 28 September 2010] [01:15:01] Quit guido_g has left this server (Client Quit).
[Tuesday 28 September 2010] [01:39:30] <CIA-20> zeromq2: 03Martin Sustrik 07maint * rf61921d 10/ src/req.cpp : REQ socket can die when reply is delivered on wrong unerlying connection – fixed -
http://bit.ly/cX2rpW[Tuesday 28 September 2010] [01:46:29] <CIA-20> zeromq2: 03Dhammika Pathirana 07maint * rc1deb22 10/ src/ypipe.hpp : crash when closing an ypipe – fixed -
http://bit.ly/bmxM8U[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Martin Sustrik 07master * rf61921d 10/ src/req.cpp : REQ socket can die when reply is delivered on wrong unerlying connection – fixed -
http://bit.ly/cX2rpW[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Dhammika Pathirana 07master * rc1deb22 10/ src/ypipe.hpp : crash when closing an ypipe – fixed -
http://bit.ly/bmxM8U[Tuesday 28 September 2010] [01:53:20] <CIA-20> zeromq2: 03Martin Sustrik 07master * r6715f9b 10/ src/ypipe.hpp :
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: Merge branch ‘maint’
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: * maint:
[Tuesday 28 September 2010] [01:53:21] <CIA-20> zeromq2: crash when closing an ypipe – fixed -
http://bit.ly/cxlL0o[Tuesday 28 September 2010] [01:54:02] Quit plol has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [01:57:13] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [03:18:51] Quit plol has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [03:56:38] Quit neopallium has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [04:03:59] Quit zmqircd has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [04:38:56] Quit keffo has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [05:20:29] <keffo> pieterh, around?
[Tuesday 28 September 2010] [05:21:35] <pieterh> keffo: hi!
[Tuesday 28 September 2010] [05:23:12] <keffo> Busy?
[Tuesday 28 September 2010] [05:24:35] <pieterh> always, but shoot...
[Tuesday 28 September 2010] [05:24:53] <keffo> I’m in the process of solidifying how the network is monitored.. Currently I gather various info & statistics in the loadbalancer which publishes regularly(~5s), and a WPF app subscribing and displaying nice graphs etc
[Tuesday 28 September 2010] [05:25:19] <pieterh> sounds good
[Tuesday 28 September 2010] [05:25:21] <keffo> but I cant really figure out a good way of limiting it.. I dont want to send a complete state every 5s
[Tuesday 28 September 2010] [05:25:22] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 28 September 2010] [05:25:30] <pieterh> how large is the state?
[Tuesday 28 September 2010] [05:26:24] <keffo> Depends, both on what info I decide to publish.. I’d like load, bandwidth usage, connected nodes and their respective stats(cpu/ram/etc), but also an overview of what is happenening, as well as more detailed info for each worker
[Tuesday 28 September 2010] [05:27:07] <pieterh> estimated size? in bytes?
[Tuesday 28 September 2010] [05:27:21] <keffo> geesh, no idea, not in the mb range at least :)
[Tuesday 28 September 2010] [05:27:40] <pieterh> if you have no idea, it’s not sensible to think about optimizing it
[Tuesday 28 September 2010] [05:27:59] <pieterh> so do a back-of-envelope calculation and come up with a figure...
[Tuesday 28 September 2010] [05:28:11] <keffo> I wondered if it was sound to have the monitoring app poll a complete current-state at startup, and then depend on a persistent sub-forwarder to handle deltastates? But that sounds very complicated
[Tuesday 28 September 2010] [05:28:48] <keffo> I can’t really guestimate, the number of nodes can range from the local setup I have here of a few machines, to much larger..
[Tuesday 28 September 2010] [05:28:57] <pieterh> ...
[Tuesday 28 September 2010] [05:29:05] <pieterh> how big is “much larger”?
[Tuesday 28 September 2010] [05:29:18] <keffo> ideally wan :)
[Tuesday 28 September 2010] [05:29:30] <pieterh> please stick a number onto it...
[Tuesday 28 September 2010] [05:29:38] <keffo> but being realistic, perhaps around 20?
[Tuesday 28 September 2010] [05:29:51] <pieterh> and how large would the state be per worker?
[Tuesday 28 September 2010] [05:30:14] <pieterh> please stick a number onto it...
[Tuesday 28 September 2010] [05:30:29] <pieterh> then multiply the two numbers and add something for the overview
[Tuesday 28 September 2010] [05:30:31] <keffo> Basic info(linpack measurements), uptime, average load, around that
[Tuesday 28 September 2010] [05:30:51] <pieterh> come back when you have a total in KB per 5 seconds, a’ight?
[Tuesday 28 September 2010] [05:32:02] <keffo> That’s not what I’m interested in, the assumption here is that no data is published that isn’t “needed”, but I would like to figure out the most efficient means of passing around that data
[Tuesday 28 September 2010] [05:32:29] <keffo> basically how to monitor a distributed system with the least amount of overhead as possible..
[Tuesday 28 September 2010] [05:32:45] <pieterh> this is for research purposes rather than an actual use case...
[Tuesday 28 September 2010] [05:32:53] <keffo> (regardless of what the data actually is)
[Tuesday 28 September 2010] [05:33:03] <keffo> well both I guess, research first, use later :)
[Tuesday 28 September 2010] [05:33:23] <pieterh> well, you can wait until Ch4 of the Guide if you want to
[Tuesday 28 September 2010] [05:33:25] <keffo> Surely this problem has been dealt with before, as loadbalancing has :)
[Tuesday 28 September 2010] [05:33:28] <pieterh> but here is how I’d do it...
[Tuesday 28 September 2010] [05:33:37] <pieterh> - maintain state in the publisher
[Tuesday 28 September 2010] [05:33:47] <pieterh> - apply updates to state and publish updates to pub socket
[Tuesday 28 September 2010] [05:34:01] <pieterh> - in subscriber, request state via req/rep socket
[Tuesday 28 September 2010] [05:34:07] <pieterh> - and also subscribe to updates
[Tuesday 28 September 2010] [05:34:10] <pieterh> - queue incoming updates
[Tuesday 28 September 2010] [05:34:22] <pieterh> - as soon as state arrives, apply updates to state and continue to do this
[Tuesday 28 September 2010] [05:34:22] <keffo> m, that’s what I was thinking as well.
[Tuesday 28 September 2010] [05:35:17] <pieterh> i think it’s robust but need to prove it
[Tuesday 28 September 2010] [05:35:17] <keffo> Things do start to get hairy if the monitor app starts to depend on delta-updates though, like “join/part” of nodes
[Tuesday 28 September 2010] [05:35:56] <keffo> you’re doing this type of stuff for ch 4?
[Tuesday 28 September 2010] [05:35:59] <pieterh> yup
[Tuesday 28 September 2010] [05:36:05] <pieterh> stateful pubsub
[Tuesday 28 September 2010] [05:36:42] <pieterh> or whatever this is properly called...
[Tuesday 28 September 2010] [05:36:55] <keffo> I’ll try it out, see if it behaves well.. :)
[Tuesday 28 September 2010] [05:37:12] <pieterh> feel free to write it up as a recipe or code sample
[Tuesday 28 September 2010] [05:37:18] <pieterh> if i can reuse that for the guide it’ll save me time
[Tuesday 28 September 2010] [05:37:37] <keffo> I’ll let you know for sure yeah
[Tuesday 28 September 2010] [05:38:06] <pieterh> the main reason for this is not so much to save network bandwidth but to allow realtime updates
[Tuesday 28 September 2010] [05:38:21] <keffo> I was aso thinking about ‘history-nodes’ as well.. something appealing about that, someone who keeps track of what’s going on, but isn’t directly part -of- the system
[Tuesday 28 September 2010] [05:38:48] <pieterh> indeed, this work could be totally outsourced to a stateful device
[Tuesday 28 September 2010] [05:38:50] <keffo> non-gonzo network monitoring
[Tuesday 28 September 2010] [05:39:08] <pieterh> with some notion of state + patches
[Tuesday 28 September 2010] [05:39:47] <pieterh> like pair/value updates
[Tuesday 28 September 2010] [05:39:52] <pieterh> hmm, nice
[Tuesday 28 September 2010] [05:39:59] <pieterh> it’s a distributed cache
[Tuesday 28 September 2010] [05:40:08] Quit mytrile has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [05:40:14] <keffo> yes, distributed tuple-storage
[Tuesday 28 September 2010] [05:40:19] <pieterh> yup, that’s the thing
[Tuesday 28 September 2010] [05:40:54] <keffo> I cant help but thinking most source control software face much of the same issues here
[Tuesday 28 September 2010] [05:40:59] <pieterh> in the general solution any node can update its cache
[Tuesday 28 September 2010] [05:41:21] <pieterh> it is a very useful general solution to state distribution
[Tuesday 28 September 2010] [05:41:27] <keffo> oh yeah, one can think of it as cache hits and misses, makes things clearer
[Tuesday 28 September 2010] [05:42:39] <pieterh> i’d start with a simple model, one publisher, many subscribers, pair/value updates
[Tuesday 28 September 2010] [05:43:05] <keffo> It reuses the same mechanisms as the rest
[Tuesday 28 September 2010] [05:44:59] <keffo> maybe I could contribute some C# stuff to go along with ch 4.. I do find it quite lacking on the site..
[Tuesday 28 September 2010] [05:45:56] <pieterh> keffo: we’ve gotten one C# example yesterday, but others would be great
[Tuesday 28 September 2010] [05:46:13] <pieterh> do start at ch1 if you could, it’s quite trivial stuff but useful to newcomers
[Tuesday 28 September 2010] [05:46:34] <keffo> yeah..
[Tuesday 28 September 2010] [05:46:50] <keffo> need a complete binding, for starters :)
[Tuesday 28 September 2010] [05:47:00] <pieterh> the binding is not complete?
[Tuesday 28 September 2010] [05:47:28] <keffo> no, there was no poll for example?
[Tuesday 28 September 2010] [05:47:34] <keffo> or did I somehow get an old version?
[Tuesday 28 September 2010] [05:48:28] <keffo> nope..
[Tuesday 28 September 2010] [05:48:53] <pieterh> who maintains this binding?
[Tuesday 28 September 2010] [05:49:00] <pieterh> ask them to fix it or submit a patch
[Tuesday 28 September 2010] [05:49:07] <keffo> not sure, says sustrik last commit I guess
[Tuesday 28 September 2010] [05:49:32] <pieterh> hmm, the owner of every project should be WRITTEN IN HUGE LETTERS
[Tuesday 28 September 2010] [05:49:41] <pieterh> otherwise it’s kind of dead by definition
[Tuesday 28 September 2010] [05:49:51] <pieterh> nzmq is something layered on top afaics
[Tuesday 28 September 2010] [05:49:58] <pieterh> different API
[Tuesday 28 September 2010] [05:50:05] <keffo> yes, but it does the same lowlevel binding of the dll
[Tuesday 28 September 2010] [05:50:44] <pieterh> a’ight
[Tuesday 28 September 2010] [05:51:10] <keffo> So merging them shouldnt be very difficult
[Tuesday 28 September 2010] [05:51:11] * pieterh is wondering who pays trolls to come talk rubbish whenever there’s a license thread
[Tuesday 28 September 2010] [05:51:22] <keffo> on the list?
[Tuesday 28 September 2010] [05:51:40] <pieterh> “What’s the latest version of ZeroMQ that’s not LGPL? I need to static link in commercial projects and the LGPL is not an option. “
[Tuesday 28 September 2010] [05:53:18] <pieterh> not a single sensible comment on that thread, just trolls coming to complain that [sic] switching to LGPL will kill 0MQ...
[Tuesday 28 September 2010] [05:53:58] <keffo> will it? =)
[Tuesday 28 September 2010] [05:54:18] <pieterh> oh, yes, of course...
[Tuesday 28 September 2010] [05:54:38] <pieterh> that’s why we have to go back in time and switch to the Microsoft Open Software License or whatever...
[Tuesday 28 September 2010] [05:55:22] <keffo> licensing is tricky business, it went and got itself hugely complicated
[Tuesday 28 September 2010] [05:55:31] <keffo> I prefer “mine” and “public domain” :)
[Tuesday 28 September 2010] [05:55:41] <pieterh> it’s not really tricky, just politically sensitive because it involves so much money
[Tuesday 28 September 2010] [05:56:13] <pieterh> every license is the contract on which the community grows
[Tuesday 28 September 2010] [05:56:21] <pieterh> LGPL and GPL are IME proven beyond a reasonable doubt to be the most effective contracts
[Tuesday 28 September 2010] [05:56:27] <pieterh> because they make it impossible to cheat
[Tuesday 28 September 2010] [05:56:34] <pieterh> end.
[Tuesday 28 September 2010] [05:56:51] <PerfDave> Not impossible, see gpl-violations.org. But very difficult ;)
[Tuesday 28 September 2010] [05:57:05] <pieterh> impossible in any sustainable sense
[Tuesday 28 September 2010] [05:57:15] <pieterh> and GPLv3 closed the loopholes people found in GPLv2
[Tuesday 28 September 2010] [05:57:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Tuesday 28 September 2010] [05:57:15] <keffo> who has the time anyway :)
[Tuesday 28 September 2010] [05:57:23] <pieterh> oh, people love to cheat
[Tuesday 28 September 2010] [05:57:41] <pieterh> but communities die when they get parasited
[Tuesday 28 September 2010] [05:58:09] <pieterh> so these trolls come and complain that LGPL will kill the community when in fact it creates it
[Tuesday 28 September 2010] [05:58:14] <pieterh> sigh.
[Tuesday 28 September 2010] [05:58:20] <keffo> :)
[Tuesday 28 September 2010] [06:03:22] <keffo> what’s the most elaborate zmq based project anyway?
[Tuesday 28 September 2010] [06:07:16] <pieterh> keffo: wow, there are some very elaborate ones out there
[Tuesday 28 September 2010] [06:07:32] <pieterh> but most are so secret that I’d have to kill you after explaining them
[Tuesday 28 September 2010] [06:07:58] <keffo> Oh I just mean scale, not impl. details :)
[Tuesday 28 September 2010] [06:08:12] <keffo> I’d like to know what level I’m at :)
[Tuesday 28 September 2010] [06:08:46] <pieterh> scale: hundreds to thousands of nodes
[Tuesday 28 September 2010] [06:09:03] <pieterh> multiple data centers
[Tuesday 28 September 2010] [06:10:00] <keffo> are they mostly about shuffling data around, or generic compute clusters?
[Tuesday 28 September 2010] [06:10:17] <pieterh> both cases
[Tuesday 28 September 2010] [06:10:47] <keffo> interesting stuff
[Tuesday 28 September 2010] [06:10:56] <pieterh> it’s also growing rapidly
[Tuesday 28 September 2010] [06:11:05] <keffo> I would assume so!
[Tuesday 28 September 2010] [06:11:16] <pieterh> the first 0MQ projects a year or two ago were maybe 10 nodes
[Tuesday 28 September 2010] [06:11:36] <pieterh> i’d say the scale is growing x10 every six months or so
[Tuesday 28 September 2010] [06:12:20] <keffo> It would be nice if the license included algorithmic contributions as opposed to solely sourcecode :)
[Tuesday 28 September 2010] [06:12:46] <pieterh> well, ideas cannot be copyrighted
[Tuesday 28 September 2010] [06:13:04] <keffo> A lot of information was probably gathered during development of those, which ideally should be shared :)
[Tuesday 28 September 2010] [06:13:12] <pieterh> well
[Tuesday 28 September 2010] [06:13:30] <pieterh> whenever possible we do move experience into the open source layers
[Tuesday 28 September 2010] [06:14:07] <pieterh> however there are often valuable business secrets in these algorithms
[Tuesday 28 September 2010] [06:14:23] <pieterh> obviously we do not consider sharing those
[Tuesday 28 September 2010] [06:14:47] <keffo> I was more thinking along the lines of “this common-practice method breaks down under these conditions” etc
[Tuesday 28 September 2010] [06:15:00] <pieterh> this is what the user guide will eventually cover
[Tuesday 28 September 2010] [06:15:36] <pieterh> at least the more common cases
[Tuesday 28 September 2010] [06:15:52] <pieterh> we can also try to document some of the higher level patterns as protocols
[Tuesday 28 September 2010] [06:15:53] <keffo> Some of the things I’ve found annoyingly void so far has been the loadbalancing(which is now covered well enough), and also recursive behaviour, which I think I’ve solved
[Tuesday 28 September 2010] [06:16:10] <pieterh> have you been using the custom routing from Ch3?
[Tuesday 28 September 2010] [06:16:30] <keffo> Yeah, pretty much, but with prioqueues
[Tuesday 28 September 2010] [06:16:37] <pieterh> what is a prioqueue?
[Tuesday 28 September 2010] [06:16:44] <keffo> both for incoming tasks and outgoing results
[Tuesday 28 September 2010] [06:16:50] <keffo> priority queue
[Tuesday 28 September 2010] [06:16:57] <pieterh> ah, so queues in your broker rather than using the socket queues
[Tuesday 28 September 2010] [06:17:03] <keffo> indeed
[Tuesday 28 September 2010] [06:17:17] <pieterh> i just didn’t want to start mucking with data structures
[Tuesday 28 September 2010] [06:17:23] <pieterh> but I think it’s inevitable
[Tuesday 28 September 2010] [06:17:41] <pieterh> i already had to define a zmsg class, will probably define a zqueue class as well
[Tuesday 28 September 2010] [06:17:51] <keffo> messages and tasks have priorities, as well as workers based on a mix of scimark(hardware) and something like a running average network ‘behaviour’
[Tuesday 28 September 2010] [06:17:57] <pieterh> right
[Tuesday 28 September 2010] [06:18:40] <keffo> Another thing I’ve been struggling to figure out a ‘pretty’ solution to is how to make workers present their updates when doing long running jobs..
[Tuesday 28 September 2010] [06:18:58] <pieterh> intermediate updates?
[Tuesday 28 September 2010] [06:18:59] <keffo> That’ sortof ties into what we talked about earlier with the Monitoring app
[Tuesday 28 September 2010] [06:19:00] <keffo> yeah
[Tuesday 28 September 2010] [06:19:06] <pieterh> two threads in the workers, I assume
[Tuesday 28 September 2010] [06:19:17] Quit neopallium has left this server (Read error: Operation timed out).
[Tuesday 28 September 2010] [06:19:17] <pieterh> workers as micro clusters
[Tuesday 28 September 2010] [06:19:31] <pieterh> it’s fractal :-)
[Tuesday 28 September 2010] [06:19:37] <pieterh> every node can be a cluster of nodes
[Tuesday 28 September 2010] [06:19:40] <keffo> leaving it solely up to the designer of the job (ie, they post progress at will) leads to abuse most likely
[Tuesday 28 September 2010] [06:20:07] <pieterh> sounds like you’re solving a lot of interesting problems
[Tuesday 28 September 2010] [06:20:11] <keffo> indeed :)
[Tuesday 28 September 2010] [06:20:21] <keffo> most interesting of all is the fact that a job can post new jobs :)
[Tuesday 28 September 2010] [06:20:22] <pieterh> you should write about it, if you can
[Tuesday 28 September 2010] [06:20:27] <keffo> that’s a brain teaser if anything :)
[Tuesday 28 September 2010] [06:20:28] <keffo> yeah
[Tuesday 28 September 2010] [06:20:38] <keffo> It’s becoming quite large to be honest :)
[Tuesday 28 September 2010] [06:20:46] <pieterh> well, stack-based simulated recursion is an old technique
[Tuesday 28 September 2010] [06:20:50] <keffo> but it’s working well
[Tuesday 28 September 2010] [06:20:56] <pieterh> it’s how we used to do quicksort in cobol
[Tuesday 28 September 2010] [06:21:04] <keffo> stack-based?
[Tuesday 28 September 2010] [06:21:17] <pieterh> your prioqueue can also be a stack
[Tuesday 28 September 2010] [06:21:22] <pieterh> you can push jobs to the front
[Tuesday 28 September 2010] [06:21:28] <pieterh> or to the back
[Tuesday 28 September 2010] [06:21:32] <keffo> ah yeah, that’s the priority of the messages :)
[Tuesday 28 September 2010] [06:21:41] <pieterh> that’s how you simulate recursion
[Tuesday 28 September 2010] [06:21:43] <keffo> he deeper the recursion, the higher the priority
[Tuesday 28 September 2010] [06:21:58] <pieterh> priority is perhaps the wrong metaphor
[Tuesday 28 September 2010] [06:22:12] <pieterh> in fact it’s “push these child jobs” followed by “pop next job and execute”
[Tuesday 28 September 2010] [06:24:09] <pieterh> If you look at section 3, you see how Quicksort (recursion) works using a stack
[Tuesday 28 September 2010] [06:24:33] <keffo> My solution was to have each worker-node (which owns one worker-process per cpu-core) can request to have an additional worker spawned while it sleeps.. So when it posts a child-job, it does so with a higher priority
[Tuesday 28 September 2010] [06:24:50] <keffo> interesting
[Tuesday 28 September 2010] [06:25:25] <pieterh> Leif and I developed these techniques in the 80’s...
[Tuesday 28 September 2010] [06:25:35] <keffo> how old are you? =)
[Tuesday 28 September 2010] [06:25:38] <pieterh> it’s very basic but I think it maps correctly to recursive messaging
[Tuesday 28 September 2010] [06:25:47] <pieterh> not so old
[Tuesday 28 September 2010] [06:25:51] <pieterh> :-)
[Tuesday 28 September 2010] [06:25:58] <pieterh> 47, to be accurate
[Tuesday 28 September 2010] [06:26:17] <keffo> that was grad student times then? =)
[Tuesday 28 September 2010] [06:26:34] <pieterh> nope, first job developing tools for large software houses
[Tuesday 28 September 2010] [06:26:45] <pieterh> event-driven concurrency in cobol
[Tuesday 28 September 2010] [06:27:35] <keffo> obol of all things :)
[Tuesday 28 September 2010] [06:27:57] <keffo> there’s a scary amount of need for cobol developers now
[Tuesday 28 September 2010] [06:28:36] <pieterh> lol...
[Tuesday 28 September 2010] [06:28:54] <pieterh> we used to train people to become cobol developers in like 3 weeks
[Tuesday 28 September 2010] [06:29:13] <keffo> That’s even more scary :)
[Tuesday 28 September 2010] [06:29:22] <keffo> like todays php crowd I guess?
[Tuesday 28 September 2010] [06:29:29] <pieterh> hmm, yeah, I guess
[Tuesday 28 September 2010] [06:29:53] <pieterh> Cobol was good for mediocre programmers, they could make stuff that worked, and didn’t kill the system
[Tuesday 28 September 2010] [06:30:13] <keffo> I’d be very afraid if my bank announced they moved from cobol to php :)
[Tuesday 28 September 2010] [06:30:26] <pieterh> indeed
[Tuesday 28 September 2010] [06:30:48] <keffo> or voting/pacemakers :)
[Tuesday 28 September 2010] [06:44:44] Quit guido_g has left this server (Quit: Boom...).
[Tuesday 28 September 2010] [06:57:18] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [07:01:47] Quit keffo has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [07:28:03] Quit bbigras has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [07:33:31] <CIA-20> jzmq: 03Stefan Majer 07master * rd46166f 10/ src/org/zeromq/ZMQ.java : Merged from upstream -
http://bit.ly/c9pkwW[Tuesday 28 September 2010] [07:33:31] <CIA-20> jzmq: 03Stefan Majer 07master * r05b384d 10/ src/org/zeromq/ZMQ.java : Reduced duplicate Javadoc comments by references to the corresponding setter. -
http://bit.ly/dvaPdr[Tuesday 28 September 2010] [07:33:39] <CIA-20> jzmq: 03Stefan Majer 07master * r1a98406 10/ src/org/zeromq/ZMQ.java : References to the man pages to further clarify the Javadoc. -
http://bit.ly/c5FVwB[Tuesday 28 September 2010] [08:13:38] Join plq has joined this channel (
~plq@78.189.30.218).
[Tuesday 28 September 2010] [08:33:56] Join eintr has joined this channel (
~dln@88.80.182.68).
[Tuesday 28 September 2010] [08:40:18] Quit plq has left this server (Quit: Leaving).
[Tuesday 28 September 2010] [08:49:48] Quit mytrile has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [08:55:33] Quit plol has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [09:13:59] Part plol has left this channel.
[Tuesday 28 September 2010] [09:27:54] <CIA-20> zeromq2: 03Gonzalo Diethelm 07master * r87beaaa 10/ (14 files in 4 dirs): ZMQ_TYPE socket option added -
http://bit.ly/b3HZYe[Tuesday 28 September 2010] [09:30:58] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [09:32:23] <CIA-20> zeromq2: 03Martin Sustrik 07master * r10bb9d0 10/ AUTHORS : Dhammika Pathirana was missing from the AUTOHRS file for some reason – fixed -
http://bit.ly/cz4TZ7[Tuesday 28 September 2010] [09:51:49] Quit sbahra has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [10:07:59] Quit xla has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [10:15:38] Quit jond has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [10:39:50] <CIA-20> zeromq2: 03Steven McCoy 07master * r00cd7d4 10/ (7 files in 3 dirs): Upgrade to OpenPGM-5.0.78 -
http://bit.ly/aKqJjZ[Tuesday 28 September 2010] [10:42:13] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: 03Steven McCopy 07master * r1dc4531 10/ (4 files): (log message trimmed)
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Add assertions to check for OpenPGM calls with invalid parameters.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Assertion to check that pgm_getaddrinfo is actually returning something.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Missing pgm_connect call.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Typo on TOS causing immediate abort.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * Placeholder calls for timeouts whilst continuing spin loop functionality.
[Tuesday 28 September 2010] [11:01:02] <CIA-20> zeromq2: * OpenPGM v5 now supports reference counting so remove init checks.
[Tuesday 28 September 2010] [11:05:07] Quit zmqircd has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [11:26:22] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 28 September 2010] [11:28:03] <psino> I’m having some issues with understanding how a PUSH socket with no downstream nodes is supposed to behave. There seem to be a difference between the sockets whether they have been used as .bind or .connect
[Tuesday 28 September 2010] [11:28:53] <psino> the connect_socket sends the data (send does not block) even if there are no downstream nodes, but the bind_socket blocks
[Tuesday 28 September 2010] [11:30:48] <psino> is this the expected behaviour? from what I understand after reading
http://api.zeromq.org/zmq_socket.html is that both should have been blocking
[Tuesday 28 September 2010] [11:31:07] <psino> As its said here: “When a ZMQ_PUSH socket enters an exceptional state due to having reached the high water mark for all downstream nodes, or if there are no downstream nodes at all, then any zmq_send(3) operations on the socket shall block until the exceptional state ends or at least one downstream node becomes available for sending; messages are not discarded.”
[Tuesday 28 September 2010] [11:37:27] <sustrik> psino: yes. it works that way
[Tuesday 28 September 2010] [11:38:02] <sustrik> on connect, socket can immediately create a queue to store messages in – even before actual connection is established
[Tuesday 28 September 2010] [11:38:41] <psino> so the manual is incorrect/imprecise?
[Tuesday 28 September 2010] [11:39:10] <sustrik> when binding, the socket has to wait till peers connect, it cannot create a queue itself (it doesn’t even know whether there’ll be a connection in the future)
[Tuesday 28 September 2010] [11:39:43] <sustrik> imprecise, i would say
[Tuesday 28 September 2010] [11:39:55] <sustrik> connect assumed that there’s a peer
[Tuesday 28 September 2010] [11:40:06] <sustrik> exactly 1 peer
[Tuesday 28 September 2010] [11:40:11] <sustrik> bind makes no such assumption
[Tuesday 28 September 2010] [11:41:28] <psino> hmm
[Tuesday 28 September 2010] [11:52:05] <jashmenn> hey can anyone give me some guidance on socket migration between threads
[Tuesday 28 September 2010] [11:52:15] <jashmenn> i’m having a tough time figuring out how that works
[Tuesday 28 September 2010] [11:52:30] <jashmenn> basically i want to close a socket from a thread different than the thread that started it
[Tuesday 28 September 2010] [11:58:14] <mato> sustrik: yo
[Tuesday 28 September 2010] [11:58:48] Quit eintr has left this server (Quit: Leaving.).
[Tuesday 28 September 2010] [11:59:45] <sustrik> mato: hi
[Tuesday 28 September 2010] [11:59:57] <mato> sustrik: i would have appreciated a heads up on the OpenPGM commits
[Tuesday 28 September 2010] [12:00:09] <mato> sustrik: it does touch the build system, of which you declared me the maintainer :-)
[Tuesday 28 September 2010] [12:00:16] <sustrik> ah
[Tuesday 28 September 2010] [12:00:22] <sustrik> i blindly applied the patches :|
[Tuesday 28 September 2010] [12:00:29] <mato> also, isn’t the author of those commits steve mccoy?
[Tuesday 28 September 2010] [12:00:36] <sustrik> yes, he is
[Tuesday 28 September 2010] [12:00:47] <mato> ah, it’s there
[Tuesday 28 September 2010] [12:00:56] <mato> ‘cept you have a typo
[Tuesday 28 September 2010] [12:00:59] <mato> but that’s my fault
[Tuesday 28 September 2010] [12:01:01] <sustrik> ?
[Tuesday 28 September 2010] [12:01:07] * mato is documenting the process
[Tuesday 28 September 2010] [12:01:20] <mato> sustrik: you should not be typing author names manually if at all possible
[Tuesday 28 September 2010] [12:01:32] <sustrik> no idea how to do it automatically
[Tuesday 28 September 2010] [12:01:38] <mato> git-am :-)
[Tuesday 28 September 2010] [12:02:08] <sustrik> good god!
[Tuesday 28 September 2010] [12:02:26] <mato> sustrik: anyway, please do ask for review before committing stuff that touches other people’s designated areas of responsibility
[Tuesday 28 September 2010] [12:02:37] <sustrik> anyway, can you check the part that touches the build system post hoc?
[Tuesday 28 September 2010] [12:02:42] <mato> yes, i can, i will
[Tuesday 28 September 2010] [12:02:47] <sustrik> thanks
[Tuesday 28 September 2010] [12:02:51] <mato> but please don’t do that in future :-)
[Tuesday 28 September 2010] [12:02:54] <mato> including e.g. docs
[Tuesday 28 September 2010] [12:02:56] <sustrik> sure
[Tuesday 28 September 2010] [12:03:03] <sustrik> eek
[Tuesday 28 September 2010] [12:03:07] <mato> eek?
[Tuesday 28 September 2010] [12:03:12] <sustrik> i’ve just committed gonzalos patch
[Tuesday 28 September 2010] [12:03:22] <sustrik> which touchs docs
[Tuesday 28 September 2010] [12:03:36] <sustrik> ZMQ_TYPE socket opt description
[Tuesday 28 September 2010] [12:03:42] <mato> it also has a commit to .gitignore for bin/ for some reason
[Tuesday 28 September 2010] [12:03:51] <mato> as part of gonzalo’s patch
[Tuesday 28 September 2010] [12:03:55] <sustrik> hm
[Tuesday 28 September 2010] [12:04:12] <sustrik> it shouldn’t be there but once it’s there it doesn’t hurt
[Tuesday 28 September 2010] [12:04:15] <mato> sure
[Tuesday 28 September 2010] [12:04:24] <mato> but my point is be careful about what is in a patch
[Tuesday 28 September 2010] [12:04:43] <sustrik> what are we going to do with patches that intersect different areas of functionality?
[Tuesday 28 September 2010] [12:04:53] <sustrik> there’s no clear committer there
[Tuesday 28 September 2010] [12:05:00] <mato> ask the interested parties for review
[Tuesday 28 September 2010] [12:05:03] <mato> that’s how it normally works
[Tuesday 28 September 2010] [12:05:10] <sustrik> ok, will do
[Tuesday 28 September 2010] [12:05:35] <mato> also, if you want my attention quickly a good way to do that is to Cc: me directly as well as sending email to the list
[Tuesday 28 September 2010] [12:05:43] <sustrik> ack
[Tuesday 28 September 2010] [12:05:51] <mato> that way it lands in my INBOX rather than in the auto-filtered-away list folder :-)
[Tuesday 28 September 2010] [12:35:18] Quit bgranger has left this server (Quit: bgranger).
[Tuesday 28 September 2010] [12:42:49] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * re3e6b7f 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java): Added support for getType() and added some missing constants. -
http://bit.ly/at8wUF[Tuesday 28 September 2010] [12:43:04] Quit jsimmons has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [12:57:49] Quit psino has left this server (Quit: psino).
[Tuesday 28 September 2010] [13:02:16] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [13:03:38] Quit keffo has left this server (Ping timeout: 264 seconds).
[Tuesday 28 September 2010] [13:12:31] Quit psino has left this server (Client Quit).
[Tuesday 28 September 2010] [13:25:37] Quit keffo has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [13:30:21] <vharron> Is there a list of projects using zeromq? I’m trying to understand what it does well by looking at the problems it solves.
[Tuesday 28 September 2010] [13:52:22] <drbobbeaty> vharron: I can’t give you a list of projects, but I can tell you about mine. I’m using ZMQ as a transport layer using OpenPGM and the reliable multicast. I’m using this because what I need is a transport more than a full-fledged messaging system like Tibco or 29West. It’s working out wonderfully in this capacity and we’re moving a lot of messages through the system.
[Tuesday 28 September 2010] [13:53:36] <drbobbeaty> It’s not a real full-blown messaging system - but you can build one with it. I just see it’s real target as a very advanced transport system with a lot of flexibility.
[Tuesday 28 September 2010] [14:04:24] Quit omarkj has left this server (Quit: omarkj).
[Tuesday 28 September 2010] [14:34:33] Quit vharron has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [15:16:22] Quit gavinstark has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [15:51:58] Join jond has joined this channel (
~jon@87.74.16.24).
[Tuesday 28 September 2010] [15:55:18] Quit
mw_ has left this server (Quit:
mw_).
[Tuesday 28 September 2010] [16:20:41] Quit zirpu has left this server (Quit: leaving).
[Tuesday 28 September 2010] [16:23:14] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Tuesday 28 September 2010] [16:29:44] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [16:47:54] <CIA-20> zeromq2: 03Steven McCoy 07master * ra729357 10/ (6 files): more fixes to (e)pgm transport -
http://bit.ly/9E1o2T[Tuesday 28 September 2010] [16:50:08] Quit zmqircd has left this server (Ping timeout: 245 seconds).
*** on Tue Sep 28 16:57:48 2010
[Tuesday 28 September 2010] [16:57:48] Topic The channel topic is “Welcome!”.
[Tuesday 28 September 2010] [16:57:52] Mode Channel modes: no messages from outside, topic protection
[Tuesday 28 September 2010] [16:57:52] Created This channel was created on 2010-02-05 10:44.
[Tuesday 28 September 2010] [17:02:36] Quit zmqircd has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [17:29:16] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Tuesday 28 September 2010] [17:54:10] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [18:15:07] Quit jashmenn has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [18:28:15] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Tuesday 28 September 2010] [19:14:03] <lluad> If I have multiple clients connecting to a single server over TCP, is there a good idiom for the server to send an unsolicited message to just one of the clients?
[Tuesday 28 September 2010] [19:18:21] Quit rgl has left this server (Ping timeout: 265 seconds).
[Tuesday 28 September 2010] [19:20:50] <Zao> lluad: When connecting a client, send a connection string to the server if it needs to communicate with you?
[Tuesday 28 September 2010] [19:24:16] Quit lluad has left this server (Ping timeout: 240 seconds).
[Tuesday 28 September 2010] [19:24:16] Nick
lluad_ is now known as lluad.
[Tuesday 28 September 2010] [19:38:31] Quit lluad has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [19:38:31] Nick
lluad_ is now known as lluad.
[Tuesday 28 September 2010] [20:29:07] Quit rbancroft has left this server (Ping timeout: 252 seconds).
[Tuesday 28 September 2010] [20:34:15] <novas0x2a> is there an example for how to use zmq_poll properly? This code (where i think i’m using it right?) crashes reliably:
http://codepad.org/uyAlBSOw[Tuesday 28 September 2010] [20:35:28] <novas0x2a> oh wow, i’m lame... ignore that code for a sec..
[Tuesday 28 September 2010] [20:37:54] Quit novas0x2a has left this server (Quit: Leaving.).
[Tuesday 28 September 2010] [21:11:27] Quit iFire has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [21:12:06] Quit miguelito has left this server (Read error: Connection reset by peer).
[Tuesday 28 September 2010] [21:17:59] Quit xla has left this server (Ping timeout: 276 seconds).
[Tuesday 28 September 2010] [21:24:18] Quit iFire` has left this server (Remote host closed the connection).
[Tuesday 28 September 2010] [21:33:26] Quit
RobertLJ_ has left this server (Read error: Operation timed out).
[Tuesday 28 September 2010] [22:28:08] Part neopallium has left this channel (“Leaving”).
[Tuesday 28 September 2010] [22:45:45] Quit lluad has left this server (Quit: lluad).
[Tuesday 28 September 2010] [23:11:38] Quit RobertLJ has left this server (Ping timeout: 272 seconds).
[Tuesday 28 September 2010] [23:13:49] Quit lluad has left this server (Quit: lluad).
[Wednesday 29 September 2010] [01:11:06] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [01:14:37] Quit iFire has left this server (Remote host closed the connection).
[Wednesday 29 September 2010] [01:32:26] Quit keffo has left this server (Ping timeout: 264 seconds).
[Wednesday 29 September 2010] [01:54:54] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 29 September 2010] [02:11:08] Quit keffo has left this server (Ping timeout: 276 seconds).
[Wednesday 29 September 2010] [02:16:02] Quit rgl has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [02:19:04] Quit jond has left this server (Quit: Leaving).
[Wednesday 29 September 2010] [02:19:38] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [04:03:29] Join eintr has joined this channel (
~dln@88.80.182.68).
[Wednesday 29 September 2010] [04:09:54] <CIA-20> zeromq2: 03Ilja Golshtein 07master * r4482dd7 10/ (AUTHORS include/zmq.hpp): C++ binding destructors don’t throw exceptions -
http://bit.ly/9r4bo6[Wednesday 29 September 2010] [04:20:42] Quit iFire has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [04:59:07] Quit keffo has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [05:25:16] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 29 September 2010] [05:39:03] Join omarkj has joined this channel (
~omarkj@81.15.34.139).
[Wednesday 29 September 2010] [05:43:31] Quit keffo has left this server (Ping timeout: 252 seconds).
[Wednesday 29 September 2010] [06:04:01] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [06:17:12] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [06:34:19] Quit iFire` has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [06:39:52] Join xla has joined this channel (
~xla@89.246.67.229).
[Wednesday 29 September 2010] [06:50:58] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [07:34:27] Quit pieterh has left this server (Ping timeout: 265 seconds).
[Wednesday 29 September 2010] [07:35:42] Quit keffo has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [08:21:00] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [08:35:11] Quit iFire has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [10:00:35] Quit guido_g has left this server (Quit: Boom...).
[Wednesday 29 September 2010] [10:04:42] Quit bgranger has left this server (Ping timeout: 265 seconds).
[Wednesday 29 September 2010] [10:07:06] <llasm> hi !!
[Wednesday 29 September 2010] [10:10:12] <sustrik> llasm: hi
[Wednesday 29 September 2010] [10:49:56] Quit mytrile has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [10:51:08] <llasm> how to install zmq*
[Wednesday 29 September 2010] [10:51:15] <llasm> i need a tuto
[Wednesday 29 September 2010] [10:53:54] <keffo> define ‘install’?
[Wednesday 29 September 2010] [10:55:48] <pieterh> please tell me if anything is missing or unclear on the instructions there
[Wednesday 29 September 2010] [10:58:50] Quit bgranger has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [11:00:23] Quit eintr has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [11:19:32] Quit bgranger has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [11:21:14] Quit jsimmons has left this server (Ping timeout: 255 seconds).
[Wednesday 29 September 2010] [11:27:49] <llasm> pieterh: i follow this instructions
[Wednesday 29 September 2010] [11:28:20] <llasm> but when try python -c “import zmq”
[Wednesday 29 September 2010] [11:28:40] <llasm> i receive cannot import name _zmq
[Wednesday 29 September 2010] [11:29:29] <pieterh> llasm: ok, you need to also build the Python API
[Wednesday 29 September 2010] [11:30:55] <pieterh> did you install 0MQ into system libraries (‘sudo make install’)?
[Wednesday 29 September 2010] [11:31:23] <pieterh> then you have to specify the system libraries in setup.cfg before doing ‘python setup.py install’
[Wednesday 29 September 2010] [11:31:41] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [11:32:19] <llasm> yes i use this command ‘make install’ with root user
[Wednesday 29 September 2010] [11:33:07] Quit friendshi has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [11:33:11] <pieterh> what does your setup.cfg contain?
[Wednesday 29 September 2010] [11:33:17] <llasm> but i don’t find zeromq2-dev folder in /user/local/
[Wednesday 29 September 2010] [11:34:20] <sustrik> there’s no such folder
[Wednesday 29 September 2010] [11:34:32] <sustrik> you should see /user/local/include/zmq.h for example
[Wednesday 29 September 2010] [11:35:25] <friendshi> llasm: see the section titled ‘Linux’ under ‘Building and installation’ on this page:
http://github.com/zeromq/pyzmq[Wednesday 29 September 2010] [11:37:23] <llasm> my setup.cfg contain
[Wednesday 29 September 2010] [11:37:28] <llasm> library_dirs = /usr/local/zeromq-dev/lib
[Wednesday 29 September 2010] [11:37:32] <llasm> include_dirs = /usr/local/zeromq-dev/include
[Wednesday 29 September 2010] [11:38:58] <friendshi> try this: python setup.py build_ext –rpath=/usr/local/zeromq-dev/lib –inplace
[Wednesday 29 September 2010] [11:39:51] <friendshi> if /usr/local/zeromq-dev/lib is where you actually installed zeromq library
[Wednesday 29 September 2010] [11:40:24] <pieterh> llasm: this is a common mistake, the example setup.cfg is confusing
[Wednesday 29 September 2010] [11:40:45] <pieterh> edit it so that the paths are /usr/local/lib and /usr/local/include
[Wednesday 29 September 2010] [11:42:40] <sustrik> where’s the example?
[Wednesday 29 September 2010] [11:43:22] <pieterh> setup.cfg.template
[Wednesday 29 September 2010] [11:43:30] <sustrik> pyzmq project?
[Wednesday 29 September 2010] [11:43:35] <pieterh> yes
[Wednesday 29 September 2010] [11:43:49] <pieterh> i’m making a quickstarter for Python on the www.zeromq.com download page
[Wednesday 29 September 2010] [11:44:00] <sustrik> ok, let me create a ticket asking to change the example
[Wednesday 29 September 2010] [11:44:09] <pieterh> ack
[Wednesday 29 September 2010] [11:44:16] <llasm> i think that the installation is ok. so i’ve this error when i try : python -c ‘import zmq’
[Wednesday 29 September 2010] [11:44:17] <llasm> ImportError: libzmq.so.0: cannot open shared object file: No such file or directory
[Wednesday 29 September 2010] [11:44:20] <pieterh> also the page should not IMO tell the user to get the zeromq git
[Wednesday 29 September 2010] [11:44:21] <llasm> is normal?
[Wednesday 29 September 2010] [11:44:35] <pieterh> llasm: after zeromq install, ‘sudo ldconfig’
[Wednesday 29 September 2010] [11:45:32] <friendshi> thats the error i was getting before doing the build_ext part from the documentation
[Wednesday 29 September 2010] [11:46:29] <pieterh> friendshi, ah, thanks
[Wednesday 29 September 2010] [11:52:44] <llasm> i excute ‘ldconfig’ but i receive the same error
[Wednesday 29 September 2010] [11:53:31] <sustrik> is there zmq.h in /usr/local/include?
[Wednesday 29 September 2010] [11:55:14] <llasm> yes
[Wednesday 29 September 2010] [11:55:48] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [11:55:52] <pieterh> llasm: did you run ‘sudo python setup.py install’ again?
[Wednesday 29 September 2010] [11:56:04] <llasm> yes
[Wednesday 29 September 2010] [12:00:17] * pieterh has exhausted his pyzmq knowledge
[Wednesday 29 September 2010] [12:00:39] <sustrik> how does you setup.py look like?
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> my setup.cfg contain
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> library_dirs = /usr/local/zeromq-dev/lib
[Wednesday 29 September 2010] [12:04:46] <mato> 17:37 < llasm> include_dirs = /usr/local/zeromq-dev/include
[Wednesday 29 September 2010] [12:04:52] <mato> this would be a problem
[Wednesday 29 September 2010] [12:05:01] <mato> /usr/local/zeromq-dev will not be in a default LD_LIBRARY_PATH
[Wednesday 29 September 2010] [12:05:05] <mato> don’t do that :-)
[Wednesday 29 September 2010] [12:05:27] <mato> if you want things to just work then install everything in /usr/local
[Wednesday 29 September 2010] [12:05:47] <mato> or edit your ld.so.conf / LD_LIBRARY_PATH but normally you don’t want to do that
[Wednesday 29 September 2010] [12:06:41] Quit pieterh has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [12:12:18] Quit xla has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [12:24:58] <llasm> LD_LIBRARY_PATH is empty. it’s normal?!!
[Wednesday 29 September 2010] [12:25:19] <guido_g> yes
[Wednesday 29 September 2010] [12:26:13] <llasm> ok
[Wednesday 29 September 2010] [12:28:08] <sustrik> have you changed the setup.py?
[Wednesday 29 September 2010] [12:28:55] <sustrik> it should look like this:
[Wednesday 29 September 2010] [12:29:01] <sustrik> library_dirs = /usr/local/lib
[Wednesday 29 September 2010] [12:29:11] <sustrik> include_dirs = /usr/local/include
[Wednesday 29 September 2010] [12:29:47] <ntelford> hi guys
[Wednesday 29 September 2010] [12:30:13] <ntelford> just having a dig around the Java API and I’m wondering if there’s any support for streaming very large messages without having to buffer them in local memory first?
[Wednesday 29 September 2010] [12:30:41] <ntelford> I ask because Socket.send(byte[] msg, long flags) requires the message to be buffered as a byte array first
[Wednesday 29 September 2010] [13:08:01] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [13:24:33] Quit RobertLJ has left this server (Quit: No Ping reply in 180 seconds.).
[Wednesday 29 September 2010] [14:03:08] Quit llasm has left this server (Quit: Quitte).
[Wednesday 29 September 2010] [14:36:38] Quit dirtmcgirt has left this server (Quit: dirtmcgirt).
[Wednesday 29 September 2010] [14:47:16] Quit omarkj has left this server (Quit: omarkj).
[Wednesday 29 September 2010] [15:45:46] Quit bgranger has left this server (Quit: bgranger).
[Wednesday 29 September 2010] [16:26:43] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Wednesday 29 September 2010] [16:43:47] Quit psino has left this server (Quit: psino).
[Wednesday 29 September 2010] [16:53:04] Quit novas0x2a has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [17:31:03] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Wednesday 29 September 2010] [17:45:31] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [17:45:33] Quit RobertLJ has left this server (Ping timeout: 245 seconds).
[Wednesday 29 September 2010] [17:47:27] Quit eintr has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [17:48:12] Quit xla has left this server (Ping timeout: 272 seconds).
[Wednesday 29 September 2010] [18:31:43] Quit friendshi has left this server (Remote host closed the connection).
[Wednesday 29 September 2010] [19:34:41] Quit rgl has left this server (Ping timeout: 252 seconds).
[Wednesday 29 September 2010] [19:40:32] <novas0x2a> could someone confirm that I’m using zmq_poll correctly here?
http://codepad.org/iJuspYER I reliably get an uninitialized/corrupted message back out. I’m suspecting a zmq bug, but I’d like to rule out user error.
[Wednesday 29 September 2010] [19:42:22] Quit Tasser has left this server (Read error: Operation timed out).
[Wednesday 29 September 2010] [19:46:23] Quit chmod_work has left this server (Quit: POOF!).
[Wednesday 29 September 2010] [19:55:33] Quit
mw_ has left this server (Ping timeout: 245 seconds).
[Wednesday 29 September 2010] [20:30:18] Join novas0x2a1 has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [20:33:07] Quit novas0x2a has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [20:38:57] <acts_as> Does zeromq have to talk to other zeromq (or
*MQ) hosts? Or can I have zeromq talk to xyz proprietary packets?
[Wednesday 29 September 2010] [20:42:20] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 29 September 2010] [20:47:07] Quit novas0x2a1 has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [20:57:12] <acts_as> anyone here :(
[Wednesday 29 September 2010] [21:01:58] Quit
mw_ has left this server (Quit: Alea iactus est.).
[Wednesday 29 September 2010] [21:13:59] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Wednesday 29 September 2010] [21:14:48] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [21:52:07] Join zirpu has joined this channel (
~zirpu@nefud.org).
[Wednesday 29 September 2010] [22:21:10] Quit novas0x2a has left this server (Quit: Leaving.).
[Wednesday 29 September 2010] [22:48:37] Quit RobertLJ has left this server (Read error: Connection reset by peer).
[Wednesday 29 September 2010] [23:10:58] Quit Tasser has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [23:16:18] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
[Wednesday 29 September 2010] [23:17:43] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Wednesday 29 September 2010] [23:37:12] Quit miguelito has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [00:12:15] <jasong_at_apache> I just found out about 0mq
[Thursday 30 September 2010] [00:12:25] <jasong_at_apache> i’m like in love
[Thursday 30 September 2010] [00:19:21] Quit grahamalot has left this server (Quit: grahamalot).
[Thursday 30 September 2010] [00:22:37] Quit lluad has left this server (Quit: lluad).
[Thursday 30 September 2010] [00:30:35] <Pe_Ell> heh, yeah I want to use it but I need a quick working prototype in Perl and the current drivers are blocking...
[Thursday 30 September 2010] [00:30:53] <Pe_Ell> need to look into making an AnyEvent driver but prototype comes first... but I really like the project. :)
[Thursday 30 September 2010] [00:59:22] <lestrrat> Pe_Ell: wait till zeromq 2.1
[Thursday 30 September 2010] [00:59:41] <lestrrat> I’m going to make an AnyEvent adaptor then.
[Thursday 30 September 2010] [01:00:10] <Pe_Ell> yeah, was that you I was talking to about it through email last week?
[Thursday 30 September 2010] [01:00:23] <lestrrat> Ah, duh. must be.
[Thursday 30 September 2010] [01:00:25] <Pe_Ell> I started looking at the C code but I needed to undistract myself... :)
[Thursday 30 September 2010] [01:00:29] <lestrrat> Sounded familiar.
[Thursday 30 September 2010] [01:00:56] <Pe_Ell> 0mq was a shiny object in my time of planning...
[Thursday 30 September 2010] [01:01:07] <lestrrat> :)
[Thursday 30 September 2010] [01:01:11] <Pe_Ell> so doing my initial design testing with AnyEvent::MP
[Thursday 30 September 2010] [01:01:23] <lestrrat> I see.
[Thursday 30 September 2010] [01:01:33] <Pe_Ell> but I want to switch to 0mq after I get it working
[Thursday 30 September 2010] [01:02:34] <lestrrat> I would be willing to build a prototype based on the current master, but I’m also running YAPC::Asia, so I won’t have time to hack on it until it’s over :/
[Thursday 30 September 2010] [01:02:44] <Pe_Ell> np
[Thursday 30 September 2010] [01:03:05] <Pe_Ell> I’ve got a like a week or two of my project before I can get back to it...
[Thursday 30 September 2010] [01:03:15] <lestrrat> so maybe sometime around the end of Oct... unless $dayjob catches me.
[Thursday 30 September 2010] [01:03:16] <lestrrat> righto
[Thursday 30 September 2010] [01:03:21] <Pe_Ell> from what I read look like you can use ZMQ_FD to trigger polling
[Thursday 30 September 2010] [01:03:40] <Pe_Ell> but I didn’t get terribly far into it
[Thursday 30 September 2010] [01:03:47] <lestrrat> right. you just need to pass that FD as a IO object, and register a callback using AE::io
[Thursday 30 September 2010] [01:04:14] <lestrrat> then call ZMQ’s poll function (which I need to change the API a bit...)
[Thursday 30 September 2010] [01:04:41] <Pe_Ell> yeah I was wondering if it would be that simple or not... since they mentioned that it triggers on more than just incoming events
[Thursday 30 September 2010] [01:04:45] <Pe_Ell> so you might not have any events when it triggers
[Thursday 30 September 2010] [01:05:36] <lestrrat> well, there may be caveats I’m missing, but sounds like you should be able to poll() and just have it return immediately w/o any messages.
[Thursday 30 September 2010] [01:05:40] <lestrrat> it wouldn’t matter.
[Thursday 30 September 2010] [01:05:56] <Pe_Ell> that’s what I was hoping
[Thursday 30 September 2010] [01:05:57] <lestrrat> it /would/ matter if it /didn’t/ trigger when there’s a message.
[Thursday 30 September 2010] [01:06:07] <Pe_Ell> yeah
[Thursday 30 September 2010] [01:07:03] <lestrrat> we’ll see. we might need to write a couple of wrappers around the FDs, but as long as we can decorate it like Perl IO object... we should be fine
[Thursday 30 September 2010] [01:08:10] <Pe_Ell> yeah
[Thursday 30 September 2010] [01:35:47] Quit rgl has left this server (Ping timeout: 276 seconds).
[Thursday 30 September 2010] [01:52:19] Quit keffo has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [02:11:12] Quit sustrik has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [02:46:20] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: 03Steven McCoy 07master * r96d85b2 10/ (4 files): (log message trimmed)
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Add assertions to check for OpenPGM calls with invalid parameters.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Assertion to check that pgm_getaddrinfo is actually returning something.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Missing pgm_connect call.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Typo on TOS causing immediate abort.
[Thursday 30 September 2010] [03:12:43] <CIA-20> zeromq2: * Placeholder calls for timeouts whilst continuing spin loop functionality.
[Thursday 30 September 2010] [03:12:44] <CIA-20> zeromq2: * OpenPGM v5 now supports reference counting so remove init checks.
[Thursday 30 September 2010] [03:12:44] <CIA-20> zeromq2: 03Steven McCoy 07master * rd14be62 10/ (6 files): more fixes to (e)pgm transport -
http://bit.ly/9U1BOZ[Thursday 30 September 2010] [03:12:45] <CIA-20> zeromq2: 03Ilja Golshtein 07master * rec2e859 10/ (AUTHORS include/zmq.hpp): C++ binding destructors don’t throw exceptions -
http://bit.ly/9wp2gs[Thursday 30 September 2010] [03:14:20] Quit guido_g has left this server (Quit: Boom...).
[Thursday 30 September 2010] [03:16:48] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [03:38:04] Join eintr has joined this channel (
~dln@88.80.182.68).
[Thursday 30 September 2010] [04:01:51] Quit jasong_at_apache has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [04:06:19] Quit novas0x2a has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [04:06:43] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Thursday 30 September 2010] [04:21:50] Part jasong_at_apac-1 has left this channel.
[Thursday 30 September 2010] [05:04:03] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [05:13:16] Quit mytrile has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [05:28:20] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 30 September 2010] [05:33:55] <CIA-20> zeromq2: 03Steven McCoy 07master * reaa925e 10/ src/pgm_socket.cpp : FreeBSD doesn’t define MSG_ERRQUEUE – now it’s defined in 0MQ -
http://bit.ly/9p2Kny[Thursday 30 September 2010] [05:35:28] <CIA-20> zeromq2: 03Martin Sustrik 07master * r7881ba1 10/ src/pgm_socket.cpp : minor whitespace fix -
http://bit.ly/dm6tfs[Thursday 30 September 2010] [05:38:03] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [05:59:21] Quit omarkj has left this server (Client Quit).
[Thursday 30 September 2010] [06:12:11] Quit guido_g has left this server (Ping timeout: 276 seconds).
[Thursday 30 September 2010] [06:12:26] <
js__> hi
[Thursday 30 September 2010] [06:13:23] <
js__> im having problem with the importation of the dll into visual studio 2010
[Thursday 30 September 2010] [06:14:15] <
js__> is anybody have face this trouble ?
[Thursday 30 September 2010] [06:14:43] <
js__> for the c# implementation
[Thursday 30 September 2010] [06:16:12] Quit
js__ has left this server (Client Quit).
[Thursday 30 September 2010] [07:32:28] Quit sbahra has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [07:43:00] Part jasong_at_apache has left this channel.
[Thursday 30 September 2010] [07:45:54] Quit unenana has left this server (Client Quit).
[Thursday 30 September 2010] [07:48:58] Part jasong_at_apache has left this channel.
[Thursday 30 September 2010] [08:04:05] <CIA-20> zeromq2: 03Martin Sustrik 07master * r98fa2fa 10/ src/xrep.cpp : fix in XREP’s fair queueing mechanism -
http://bit.ly/8Zh1WS[Thursday 30 September 2010] [08:05:39] Quit pasha_bastov has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [08:05:55] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [08:05:57] Nick
pasha_bastov_ is now known as pasha_bastov.
[Thursday 30 September 2010] [08:06:23] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [08:08:00] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 30 September 2010] [08:08:13] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [08:12:04] Quit pasha_bastov has left this server (Ping timeout: 272 seconds).
[Thursday 30 September 2010] [08:24:48] Quit jsimmons has left this server (Quit: Leaving).
[Thursday 30 September 2010] [08:41:47] <mrshpot> hello, mato
[Thursday 30 September 2010] [08:55:07] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [08:59:15] Join xla has joined this channel (
~xla@89.246.67.229).
[Thursday 30 September 2010] [09:10:58] <CIA-20> zeromq2: 03Max Wolf 07maint * r79aea5f 10/ (AUTHORS src/xrep.cpp): zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/9EVmvj[Thursday 30 September 2010] [09:13:15] <CIA-20> zeromq2: 03Max Wolf 07master * r79aea5f 10/ (AUTHORS src/xrep.cpp): zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/9EVmvj[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: 03Martin Sustrik 07master * rac9b05c 10/ (AUTHORS src/xrep.cpp):
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: Merge branch ‘maint’
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: * maint:
[Thursday 30 September 2010] [09:13:21] <CIA-20> zeromq2: zmq_msg_move called on uninitialised message in xrep_t::xrecv – fixed -
http://bit.ly/dbjJBL[Thursday 30 September 2010] [09:16:23] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: 03Martin Lucina 07master * r8f9080e 10/ src/zmq.cpp :
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: zmq_poll(): Fix busy-loop if timeout is zero
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: Fix a case where zmq_poll() (poll-based version) could go off into a busy-loop
[Thursday 30 September 2010] [09:31:59] <CIA-20> zeromq2: if no revents are returned and the timeout passed in is zero. -
http://bit.ly/dq8ZsW[Thursday 30 September 2010] [09:35:58] Quit zmqircd has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [09:53:03] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [09:59:34] <mato> sustrik: are you there?
[Thursday 30 September 2010] [09:59:41] <sustrik> yes
[Thursday 30 September 2010] [09:59:50] <mato> i get an occasional assertion on master
[Thursday 30 September 2010] [09:59:56] <mato> no idea what causes it
[Thursday 30 September 2010] [10:00:02] <sustrik> what’s the assertion?
[Thursday 30 September 2010] [10:00:03] <mato> Assertion failed:
inpipe_ &&
outpipe_ (xrep.cpp:53)
[Thursday 30 September 2010] [10:01:29] <sustrik> that should not happen
[Thursday 30 September 2010] [10:01:32] <mato> will try to get a core dump
[Thursday 30 September 2010] [10:01:33] <sustrik> can we reproduce it?
[Thursday 30 September 2010] [10:01:37] <mato> it’s very random
[Thursday 30 September 2010] [10:01:40] <sustrik> hm
[Thursday 30 September 2010] [10:08:24] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [10:15:47] <mato> sustrik: ok, i have a core file... it comes out of a case with close()
[Thursday 30 September 2010] [10:15:52] <mato> sustrik: so it’s related to socket shutdown
[Thursday 30 September 2010] [10:15:59] <sustrik> ack
[Thursday 30 September 2010] [10:16:08] <sustrik> can i see the backtrace?
[Thursday 30 September 2010] [11:01:41] Quit eintr has left this server (Quit: Leaving.).
[Thursday 30 September 2010] [11:03:00] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Thursday 30 September 2010] [11:39:58] <mikko> ZMQ_TYPE support added to php bindings
[Thursday 30 September 2010] [11:40:08] <mikko> i like the changed semantics with zmq_close
[Thursday 30 September 2010] [11:40:22] <mikko> with languages like php the ‘old’ semantics cause a lot of issues
[Thursday 30 September 2010] [11:40:40] <sustrik> it caused lot of problems to everone
[Thursday 30 September 2010] [11:41:13] <sustrik> but even the new solution is not perfect, zmq_term may now block for prolonged periods of time
[Thursday 30 September 2010] [11:41:32] Quit lluad has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [11:41:49] <sustrik> the only real solution is to have 0mq in kernel space
[Thursday 30 September 2010] [11:41:52] <mato> it still needs discussion and fixing, /me would love to write all the close semantics down properly but is bogged down in work for a client
[Thursday 30 September 2010] [11:42:10] <mato> sustrik: that solution is years away, hence not a solution :)
[Thursday 30 September 2010] [11:42:34] <sustrik> sure
[Thursday 30 September 2010] [11:42:58] <mikko> zmq_close with timeout would be nice i guess
[Thursday 30 September 2010] [11:43:11] <mikko> as intermediate solution
[Thursday 30 September 2010] [11:43:16] <sustrik> yeah, SO_LINGER has to be added
[Thursday 30 September 2010] [11:43:24] <sustrik> it’s on TODO list
[Thursday 30 September 2010] [11:43:31] <mikko> im one step behind
[Thursday 30 September 2010] [11:43:38] <mikko> been on holidays in sunny istanbul
[Thursday 30 September 2010] [11:43:44] <mikko> mainly focusing on kebab and raki
[Thursday 30 September 2010] [11:43:44] <sustrik> lucky you
[Thursday 30 September 2010] [11:44:12] <mikko> how did the meeting go in london?
[Thursday 30 September 2010] [11:44:28] <sustrik> there was a tube strike exactly that day :)
[Thursday 30 September 2010] [11:44:41] <sustrik> the only one who managed to get there was jon dyte
[Thursday 30 September 2010] [11:45:07] <sustrik> pieter was more lucky with chicago meetup
[Thursday 30 September 2010] [11:45:13] <mikko> do you come to london often?
[Thursday 30 September 2010] [11:45:20] <sustrik> depends
[Thursday 30 September 2010] [11:45:35] <sustrik> you’ve been in amsterdam last time, right?
[Thursday 30 September 2010] [11:45:39] <mikko> yeah
[Thursday 30 September 2010] [11:45:52] <sustrik> gonzalo just proposed another meetup in amsterdam
[Thursday 30 September 2010] [11:45:59] <sustrik> won’t you be around by chance?
[Thursday 30 September 2010] [11:46:00] <mikko> yeah, i saw the email
[Thursday 30 September 2010] [11:46:08] <mikko> it looks like im in utrecht on 18th onwards
[Thursday 30 September 2010] [11:46:15] <mikko> which is short way from amsterdam
[Thursday 30 September 2010] [11:46:21] <mikko> s/on/from/
[Thursday 30 September 2010] [11:46:29] <sustrik> nice
[Thursday 30 September 2010] [11:46:34] <mikko> that’s week early though
[Thursday 30 September 2010] [11:46:53] <mikko> it’s either one week or two week job
[Thursday 30 September 2010] [11:47:02] <sustrik> i see
[Thursday 30 September 2010] [11:47:32] <sustrik> well, let’s see whether some other folks are interested in the meetup
[Thursday 30 September 2010] [11:47:59] <mikko> amsterdam is pretty easy to get from most of the europe
[Thursday 30 September 2010] [11:48:05] <mikko> 45 mins flight from here
[Thursday 30 September 2010] [11:48:19] <sustrik> ack
[Thursday 30 September 2010] [11:50:47] <mikko> i would be keen on meet-up if an opportunity comes up
[Thursday 30 September 2010] [11:51:20] <mikko> hmmm, i get a quota of conference days to spend from work
[Thursday 30 September 2010] [11:51:30] <mikko> are you talking in any upcoming event?
[Thursday 30 September 2010] [11:53:12] <sustrik> nope, it’s just a beer event
[Thursday 30 September 2010] [11:54:08] <sustrik> i thought that if 5-6 people would come there it would be worth of spending a day in amsterdam
[Thursday 30 September 2010] [11:54:17] <sustrik> but it’s just social
[Thursday 30 September 2010] [11:55:22] <sustrik> btw, one think we’ve talked about with jon dyte is whether it wouldn’t make sense to organise a small 0mq meetup/conference
[Thursday 30 September 2010] [11:55:58] Quit keffo has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [11:56:09] <mikko> that would be nice
[Thursday 30 September 2010] [11:56:31] <mikko> varnish people do those occasionally
[Thursday 30 September 2010] [11:56:32] <sustrik> it’s purely theoretical so far
[Thursday 30 September 2010] [11:56:40] <mikko> they also brainstorm on features etc
[Thursday 30 September 2010] [11:56:57] <sustrik> ack
[Thursday 30 September 2010] [11:57:05] <sustrik> the logistics is obvious problem
[Thursday 30 September 2010] [11:57:14] <sustrik> people are all around the globe
[Thursday 30 September 2010] [11:57:28] <mikko> that kind of thing
[Thursday 30 September 2010] [11:57:53] <sustrik> yes, something like that
[Thursday 30 September 2010] [11:57:55] <mikko> maybe we could try to organise it around a larger conference
[Thursday 30 September 2010] [11:58:02] <sustrik> one option would be to hold it after fossdem
[Thursday 30 September 2010] [11:58:05] <sustrik> in brussels
[Thursday 30 September 2010] [11:58:11] <sustrik> that’s a really big conference
[Thursday 30 September 2010] [11:58:24] <sustrik> some 2000 attendeed or so
[Thursday 30 September 2010] [11:58:46] <sustrik> held jan/feb iirc
[Thursday 30 September 2010] [11:59:29] <mikko> feb it seems
[Thursday 30 September 2010] [11:59:38] <mikko> they give rooms to distribution miniconferences
[Thursday 30 September 2010] [11:59:42] <mikko> conferences*
[Thursday 30 September 2010] [11:59:54] <mikko> there was no typo actually
[Thursday 30 September 2010] [12:00:05] <mikko> anyway, it seems like a sensible idea
[Thursday 30 September 2010] [12:00:22] <mikko> that way people can attend fossdem and probably get their companies to pay for flights / accomodation
[Thursday 30 September 2010] [12:00:34] <sustrik> there are imatix offices in brussels, so it would be easy
[Thursday 30 September 2010] [12:00:42] <mikko> We hereby invite ideas and proposals for main speakers, and proposals for developer rooms. The call for stands and lightning talks will be done after announcing the accepted devrooms.
[Thursday 30 September 2010] [12:00:54] <mikko> they also provide dev rooms
[Thursday 30 September 2010] [12:01:19] <sustrik> but pieter has a fridge with beer at imatix office :)
[Thursday 30 September 2010] [12:01:37] <mikko> sounds like a plan
[Thursday 30 September 2010] [12:01:45] <mikko> mussels in brussels
[Thursday 30 September 2010] [12:01:49] <sustrik> right
[Thursday 30 September 2010] [12:02:11] <mikko> maybe fire the idea to mailing list and see if people pick up?
[Thursday 30 September 2010] [12:03:00] <omarkj> Oh, a reason to go to Brussels?
[Thursday 30 September 2010] [12:05:23] <mikko> it would be nice to brainstorm and bounce around ideas on how people use zeromq
[Thursday 30 September 2010] [12:07:30] <sustrik> yes, if people are intersted we can arrange that
[Thursday 30 September 2010] [12:13:09] <mikko> seems like pfsense uses the zmq php bindings
[Thursday 30 September 2010] [12:13:39] Quit mrshpot has left this server (Quit: Leaving).
[Thursday 30 September 2010] [12:13:50] <sustrik> what’s that?
[Thursday 30 September 2010] [12:13:55] Quit vagn has left this server (Quit: Ex-Chat).
[Thursday 30 September 2010] [12:13:59] <mikko> it’s a firewall / captive portal
[Thursday 30 September 2010] [12:14:15] <sustrik> interesting
[Thursday 30 September 2010] [12:14:26] <mikko> “pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router.”
[Thursday 30 September 2010] [12:14:42] <mikko> it seems they use zeromq to distribute config changes
[Thursday 30 September 2010] [12:14:42] <sustrik> any idea what do they use it for?
[Thursday 30 September 2010] [12:14:47] <sustrik> i see
[Thursday 30 September 2010] [12:15:15] <sustrik> actually, i would like to see 0MQ devices running on routers one day
[Thursday 30 September 2010] [12:15:17] <mikko> i guess that enables centralised management for firewall clusters
[Thursday 30 September 2010] [12:15:33] <sustrik> but we are not that fat yet
[Thursday 30 September 2010] [12:15:38] <sustrik> far*
[Thursday 30 September 2010] [12:16:40] <sustrik> the idea is that a device sits at the edge of the network (in router) and redistributes messages to the local network
[Thursday 30 September 2010] [12:17:41] <mikko> i think some of the concepts in zeromq are hard to grasp
[Thursday 30 September 2010] [12:17:44] <mikko> for some people
[Thursday 30 September 2010] [12:17:54] <mikko> well, not zeromq specific issue
[Thursday 30 September 2010] [12:18:01] <mikko> more like messaging oriented approach
[Thursday 30 September 2010] [12:18:18] <mikko> and some people seem to insist on ‘persistent’ messaging
[Thursday 30 September 2010] [12:18:44] <sustrik> yes, i am aware of that
[Thursday 30 September 2010] [12:18:51] <sustrik> we need to educate the crowd
[Thursday 30 September 2010] [12:56:19] <jasong_at_apache> one my main dev machine which is OSX (production is ubuntu) i’m getting: ./configure: line 15263: syntax error near unexpected token
`newline’
[Thursday 30 September 2010] [12:56:20] <jasong_at_apache> ./configure: line 15263: ` PKG_CHECK_MODULES(‘
[Thursday 30 September 2010] [12:56:48] <jasong_at_apache> this is optionless ./autogen.sh , ./configure (stock as is)
[Thursday 30 September 2010] [12:58:28] <jasong_at_apache> update: i just found the doc page....
[Thursday 30 September 2010] [12:59:03] <cremes> jasong_at_apache: make sure you have libtool installed on your machine
[Thursday 30 September 2010] [12:59:17] <cremes> if you use macports, do a “port install libtool” and try again
[Thursday 30 September 2010] [12:59:35] <jasong_at_apache> i do, will try! thx cremes
[Thursday 30 September 2010] [13:05:20] Quit xla has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [13:25:05] Quit lluad has left this server (Client Quit).
[Thursday 30 September 2010] [14:00:58] Quit friendshi has left this server (Quit: friendshi).
[Thursday 30 September 2010] [14:16:11] <jasong_at_apache> another issue, now with jzmq
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> libtool: compile: g++ -DHAVE_CONFIG_H -I. -D_REENTRANT -D_THREAD_SAFE -I/Library/Java/Home/include -Wall -g -O2 -MT libjzmq_la-Socket.lo -MD -MP -MF .deps/libjzmq_la-Socket.Tpo -c Socket.cpp -fno-common -DPIC -o .libs/libjzmq_la-Socket.o
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> Socket.cpp: In function ‘jlong Java_org_zeromq_ZMQ_00024Socket_getLongSockopt(JNIEnv*, _jobject*, jint)’:
[Thursday 30 September 2010] [14:16:26] <jasong_at_apache> Socket.cpp:103: error: ‘ZMQ_TYPE’ was not declared in this scope
[Thursday 30 September 2010] [14:21:09] <cremes> jasong_at_apache: i don’t think the “official” bindings have been updated to work with 2.1 yet
[Thursday 30 September 2010] [14:21:25] <cremes> if you are just starting to experiment, you might want to stick with 2.0.9
[Thursday 30 September 2010] [14:21:58] <jasong_at_apache> ah
[Thursday 30 September 2010] [14:23:02] <jasong_at_apache> sorry I’m coming across as a total noob, I am monitoring several builds figure i’d just hit you guys up for the obvious
[Thursday 30 September 2010] [14:29:11] Quit omarkj has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [14:30:20] Nick n is now known as Guest28979.
[Thursday 30 September 2010] [14:31:38] <Guest28979> I want to setup a pub-sub system where a client sends a message to a central server that does some authentication and error checking and then publishes that message to all clients.
[Thursday 30 September 2010] [14:32:11] <omarkj> Okay.
[Thursday 30 September 2010] [14:32:24] <Guest28979> I’m doing a request/response pair to send the message to the server, and pub/sub to send out the messages, and it’s working beautifully, except...
[Thursday 30 September 2010] [14:33:00] <Guest28979> These are dynamic hosts, and on ocassion, the server will die and move to a different machine.
[Thursday 30 September 2010] [14:33:42] <Guest28979> Is there an elegant way to do this, or do I need to do non-blocking recv’s with timeouts, and a heartbeat on the PUB/SUB side of things?
[Thursday 30 September 2010] [14:35:54] <Peaceful> Is this a common problem? I’m trying to install zmq and pyzmq for the first time on my OS X 10.6 box, and it looked like it worked, but I get: ImportError: cannot import name _zmq –> when I do “import zmq”
[Thursday 30 September 2010] [14:36:49] <Peaceful> I just downloaded 2.0.9, untarred it, ran ./configure, make, make install ...
[Thursday 30 September 2010] [14:37:05] <Guest28979> Peaceful: How did you install pyzmq?
[Thursday 30 September 2010] [14:37:13] <Guest28979> easy_install?
[Thursday 30 September 2010] [14:37:25] <Peaceful> then downloaded pyzmq (from git), copied setup.cfg (which looked correct), and did ‘sudo python setup.py install’
[Thursday 30 September 2010] [14:37:57] <Guest28979> What was the output of that?
[Thursday 30 September 2010] [14:40:06] * Peaceful is gathering output for pastebin
[Thursday 30 September 2010] [14:45:30] <Peaceful>
http://pastebin.com/8ct0gsif <– and here’s the full output from my ”./configure”, “make”, “sudo make install”
[Thursday 30 September 2010] [14:45:55] <Peaceful> There’s warnings, but it appeared that zmq and pyzmq both successfully installed...but doesn’t work.
[Thursday 30 September 2010] [14:46:44] <guido_g> might be a path issue
[Thursday 30 September 2010] [14:47:10] <Guest28979> I’m hardly a Mac guy, so I’m stabbing in the dark here...
[Thursday 30 September 2010] [14:47:20] <guido_g> dito.
[Thursday 30 September 2010] [14:48:17] <Peaceful> Hmm. There were no mac-specific install instructions for zmq...I checked “INSTALL”. I wonder if zmq itself works okay...
[Thursday 30 September 2010] [14:48:53] <Guest28979> Do you have a directory that looks something like ‘/Library/Python/2.6/site-packages/pyzmq-2.0.9dev-py2.6.egg’?
[Thursday 30 September 2010] [14:49:24] <Peaceful> no, but...
[Thursday 30 September 2010] [14:49:43] <Peaceful> there is this, which is close:
[Thursday 30 September 2010] [14:49:45] <Peaceful> ls /Library/Python/2.6/site-packages/zmq/
[Thursday 30 September 2010] [14:49:49] <Peaceful> __init__.py __init__.pyc _zmq.pxd _zmq.so allocate.pxd buffers.pxd devices.so eventloop log tests
[Thursday 30 September 2010] [14:49:52] <Guest28979> It’s sticking an egg.info file into /Library/Python/2.6/site-package, but where is the egg itself?
[Thursday 30 September 2010] [14:50:05] <Guest28979> Ah, OK.
[Thursday 30 September 2010] [14:50:15] <Guest28979> What’s the mac equivalent of ldconfig?
[Thursday 30 September 2010] [14:50:22] <Peaceful> There’s this: /Library/Python/2.6/site-packages/pyzmq-2.0.9dev-py2.6.egg-info
[Thursday 30 September 2010] [14:50:23] <Guest28979> Try sudo ldconfig
[Thursday 30 September 2010] [14:50:42] <Peaceful> hey!
[Thursday 30 September 2010] [14:50:52] <Peaceful> before I tried that, I thought, maybe I should try a new terminal
[Thursday 30 September 2010] [14:50:58] <Peaceful> opening up a new terminal window worked
[Thursday 30 September 2010] [14:51:02] <Guest28979> Yeah, and the new one has the path.
[Thursday 30 September 2010] [14:51:06] <Peaceful> I didn’t run ldconfig
[Thursday 30 September 2010] [14:51:16] <Guest28979> Or the python path is rebuilt, or what-have-you.
[Thursday 30 September 2010] [14:51:22] <Peaceful> ah, I bet “hash -r” would have fixed it in my original one...
[Thursday 30 September 2010] [14:51:32] <Peaceful> er, nope
[Thursday 30 September 2010] [14:51:39] <Peaceful> okay, I’ll just stick with the new window :)
[Thursday 30 September 2010] [14:51:41] <Guest28979> guido_g was right, it was a path issue.
[Thursday 30 September 2010] [14:51:49] <guido_g> YES!
[Thursday 30 September 2010] [14:51:54] <Guest28979> +1
[Thursday 30 September 2010] [14:51:56] <guido_g> where’S the cookie? :)
[Thursday 30 September 2010] [14:51:58] <Peaceful> guido_g++
[Thursday 30 September 2010] [14:52:05] <Peaceful> ^– cookie
[Thursday 30 September 2010] [14:52:23] * Peaceful hands guido_g his cookie and says “good boy!”
[Thursday 30 September 2010] [14:52:28] <Peaceful> thanks!
[Thursday 30 September 2010] [14:52:32] Nick Guest28979 is now known as nedjl.
[Thursday 30 September 2010] [14:52:46] <nedjl> Let’s see if nickserv lets me have this one.
[Thursday 30 September 2010] [14:52:51] <rgl> how good is zeromq on windows?
[Thursday 30 September 2010] [14:52:52] <Peaceful> Now I’ll go see if I can actually get zmq working. I only discovered its existence last night.
[Thursday 30 September 2010] [14:52:52] * guido_g eats cookie
[Thursday 30 September 2010] [14:53:20] * Peaceful heads off to explore zmq...and eat lunch
[Thursday 30 September 2010] [14:53:22] Quit Peaceful has left this server (Quit: Peaceful).
[Thursday 30 September 2010] [14:53:32] <mikko> rgl: should be fine
[Thursday 30 September 2010] [14:53:33] <nedjl> Peaceful: There’s a “It’s not working” workflow chart that’s very good. Most of my experience with it has involved it just working.
[Thursday 30 September 2010] [14:53:46] <mikko> rgl: as far as i know, cant really vouch for it as i dont use windows
[Thursday 30 September 2010] [14:54:20] <rgl> are there any official windows binaries?
[Thursday 30 September 2010] [14:54:26] * rgl didn’t found them on the homepage
[Thursday 30 September 2010] [14:54:53] <nedjl> I figured out a better way of asking my question – is there a way to get an error when an endpoint goes down? I don’t want zeromq to hide me from this, I need to redo some service discovery to find out where I should connect to instead.
[Thursday 30 September 2010] [14:55:19] <mikko> rgl: you should be able to build easily
[Thursday 30 September 2010] [14:55:38] <rgl> mikko, ok. will do. thx! :)
[Thursday 30 September 2010] [14:55:41] <mikko> rgl: are you looking to use one of the language bindings?
[Thursday 30 September 2010] [14:55:47] <rgl> mikko, C#
[Thursday 30 September 2010] [14:56:19] <guido_g> nedjl: nope, mq is notorously underprovisioned with this kind of stuff
[Thursday 30 September 2010] [14:57:10] <guido_g> it _might_ be that you get a notification through the logging interface in the future, the distant future...
[Thursday 30 September 2010] [14:57:42] <mikko> nedjl: at the moment you would probably implement some sort of heartbeat
[Thursday 30 September 2010] [14:58:21] <mikko> it’s getting busy in here, 66 people
[Thursday 30 September 2010] [14:59:10] <guido_g> complicated by the fact that you don’t know which connection is alive and that there is no way to use a specific underlying connection
[Thursday 30 September 2010] [15:00:25] <nedjl> mikko, guido_g: OK. My current plan to figure out if the sub/pub socket is still alive to send out a heartbeat every few seconds, and kill off all the clients if there is no response.
[Thursday 30 September 2010] [15:00:58] <guido_g> if you have use-cases involving this kind of information, please take the time to mention it on the mailing list
[Thursday 30 September 2010] [15:01:12] <nedjl> The req/res socket I’m polling on the response to a req, and if I don’t get anything after a reasonable amount of time, I redo the service discovery and try again with the new info.
[Thursday 30 September 2010] [15:02:09] Quit omarkj has left this server (Quit: omarkj).
[Thursday 30 September 2010] [15:02:20] <mikko> indeed the hearbeat is problematic
[Thursday 30 September 2010] [15:02:31] <nedjl> I’ve got a server that validates messages indicating events, and then spreads em around to anyone interested. That server will go down from time to time, and I want all the clients to notice and redo the service discovery if appropriate.
[Thursday 30 September 2010] [15:02:38] <mikko> especially in scenarios where for example a specific port is blocked accidentally on a firewall
[Thursday 30 September 2010] [15:02:53] <mikko> as the hearbeat might be going through a different port than main connection
[Thursday 30 September 2010] [15:03:31] <nedjl> These are actually a cluster of EC2 servers, and I’m sending out notifications about changes in the cluster’s state using this mechanism.
[Thursday 30 September 2010] [15:04:37] <mikko> nedjl: and you dont know the peers addresses beforehand?
[Thursday 30 September 2010] [15:04:47] <nedjl> I can’t use multicast, even though it is probably the right solution, because EC2 doesn’t support multicast without some gross tunneling hacks.
[Thursday 30 September 2010] [15:05:06] <mikko> nedjl: doesnt pgm encapsulate the multicast into tcp?
[Thursday 30 September 2010] [15:05:15] <guido_g> no
[Thursday 30 September 2010] [15:05:28] <mikko> even with tcp transport?
[Thursday 30 September 2010] [15:05:34] <nedjl> Machines will get added, machines will get killed. If the machine that gets killed is the one with the server, it’ll get started up somewhere else, and everybody will just do a lookup to find the new location.
[Thursday 30 September 2010] [15:05:36] <guido_g> no
[Thursday 30 September 2010] [15:06:06] <mikko> i might’ve misunderstood pgm in that case
[Thursday 30 September 2010] [15:07:05] <guido_g> pgm is working on ip level, ephm encapsulates it in udp
[Thursday 30 September 2010] [15:07:07] <mikko> time to read rfc 3028
[Thursday 30 September 2010] [15:07:11] <guido_g>
*epgm
[Thursday 30 September 2010] [15:07:18] <mikko> ok
[Thursday 30 September 2010] [15:07:26] <mikko> i thought you could encapsulate inside tcp as well
[Thursday 30 September 2010] [15:07:45] <guido_g> you can’t do real multicast on tcp
[Thursday 30 September 2010] [15:08:18] <nedjl> mikko: 1.1 of RFC 3028 says ” PGM runs over a datagram multicast protocol such as IP multicast [5]. “
[Thursday 30 September 2010] [15:08:34] <nedjl> Curses.
[Thursday 30 September 2010] [15:08:38] <mikko> reading 3028 now
[Thursday 30 September 2010] [15:09:39] <mikko> i guess using tcp wouldnt make much sense indeed
[Thursday 30 September 2010] [15:10:08] <mikko> does epgm work in EC2?
[Thursday 30 September 2010] [15:10:18] <nedjl> So it seems inevitable that I’m looking at lots of NOBLOCK and timeouts.
[Thursday 30 September 2010] [15:10:26] <guido_g> no, because you said multicast doesn’t work
[Thursday 30 September 2010] [15:10:27] <nedjl> Lemme lok.
[Thursday 30 September 2010] [15:10:48] <mikko> guido_g: i don’t know anything about EC2
[Thursday 30 September 2010] [15:10:57] * guido_g knows even less
[Thursday 30 September 2010] [15:11:05] <mikko> but i would imagine there are use-cases for multicast in ‘cloud’
[Thursday 30 September 2010] [15:11:10] <guido_g> <nedjl> I can’t use multicast, even though it is probably the right solution, because EC2 doesn’t support multicast without some gross tunneling hacks.
[Thursday 30 September 2010] [15:11:41] <guido_g> mikko: definitely, but it’s also not easy and might even kill the network
[Thursday 30 September 2010] [15:11:45] <mikko> it might be hard to organize safely
[Thursday 30 September 2010] [15:11:59] <guido_g> so it is wise to disable it
[Thursday 30 September 2010] [15:12:33] <mikko> what’s the current best approach to heartbeats with zmq?
[Thursday 30 September 2010] [15:12:40] <mikko> use a separate oob socket for heartbeat?
[Thursday 30 September 2010] [15:12:50] <nedjl> mikko: That’s probably the question I should be asking.
[Thursday 30 September 2010] [15:12:53] <guido_g> i don’t think that there is a an easy and useful way for real multicast on these sold virtual machines
[Thursday 30 September 2010] [15:13:23] <guido_g> if you use a separate socket your testing this one, not the other one
[Thursday 30 September 2010] [15:13:47] <mikko> guido_g: i would imagine it would also require end to end encryption for the multicast
[Thursday 30 September 2010] [15:14:16] <mikko> i guess it depends on how the cloud is organised, i remember some providers have private clouds with separate VLANs
[Thursday 30 September 2010] [15:14:37] <guido_g> mikko: yes, and every one would be forced to use it... no way
[Thursday 30 September 2010] [15:15:06] <guido_g> won’t work on the “machine on demand” front
[Thursday 30 September 2010] [15:15:07] <mikko> guido_g: not forced of course, but if they cared for the authenticity of the data then yes
[Thursday 30 September 2010] [15:15:29] <mikko> i guess thats almost forcing
[Thursday 30 September 2010] [15:15:56] <guido_g> otoh multicast steals bandwidth from other users, if a subscriber is in their network segment
[Thursday 30 September 2010] [15:16:05] <guido_g> so one has to deal with that too
[Thursday 30 September 2010] [15:16:31] <guido_g> i don’t think real multicast will become widely used in to vapor... oops... cloud world
[Thursday 30 September 2010] [15:16:41] <mikko> and non rate-limited multicast would be a bit scary
[Thursday 30 September 2010] [15:21:40] <jasong_at_apache> anyone here successfully compile jzmq on osx?
[Thursday 30 September 2010] [15:23:18] Quit nedjl has left this server (Quit: Page closed).
[Thursday 30 September 2010] [15:26:50] <mikko> jasong_at_apache: what is the problem you are facing?
[Thursday 30 September 2010] [15:26:55] <mikko> i can give it a spin
[Thursday 30 September 2010] [15:27:18] <jasong_at_apache> on 10.6 i’ve successfully compiled 2.0.9 (download area)
[Thursday 30 September 2010] [15:27:42] <jasong_at_apache> yet with jzmq i get: Socket.cpp: In function ‘jlong Java_org_zeromq_ZMQ_00024Socket_getLongSockopt(JNIEnv*, _jobject*, jint)’:
[Thursday 30 September 2010] [15:27:42] <jasong_at_apache> Socket.cpp:103: error: ‘ZMQ_TYPE’ was not declared in this scope
[Thursday 30 September 2010] [15:27:51] <jasong_at_apache> during make
[Thursday 30 September 2010] [15:27:53] <mikko> ZMQ_TYPE is in 2.1 only
[Thursday 30 September 2010] [15:28:19] <mikko> and i think the type is ‘int’ iirc
[Thursday 30 September 2010] [15:28:20] <jasong_at_apache> i thought i did a checkout -b from a previous commit (i need to do that again)
[Thursday 30 September 2010] [15:28:31] <jasong_at_apache> how far back do i go? there are no tags atm
[Thursday 30 September 2010] [15:28:34] <mikko> the method name suggests that its a long
[Thursday 30 September 2010] [15:28:53] <mikko> jasong_at_apache: 2.1.0 zmq i mean
[Thursday 30 September 2010] [15:28:58] <mikko> jasong_at_apache: the current github trunk
[Thursday 30 September 2010] [15:29:23] <jasong_at_apache> so it’s not worth using the binding with the maint branch?
[Thursday 30 September 2010] [15:29:29] <sustrik> the change to jzmq was committed by gonzalo today
[Thursday 30 September 2010] [15:29:49] <sustrik> so if you grab yesteday’s version it should work with 2.0.9
[Thursday 30 September 2010] [15:30:01] <jasong_at_apache> ah
[Thursday 30 September 2010] [15:30:02] <mikko> jasong_at_apache: if you edit Socket.cpp line 103
[Thursday 30 September 2010] [15:30:13] <jasong_at_apache> and remove it
[Thursday 30 September 2010] [15:30:13] <mikko> #ifdef ZMQ_TYPE around the line should help
[Thursday 30 September 2010] [15:30:23] <jasong_at_apache> and i’ll grab the previous commit
[Thursday 30 September 2010] [15:30:32] <mikko> sustrik: ZMQ_TYPE is ‘int’ ?
[Thursday 30 September 2010] [15:30:32] <sustrik> mikko: yes
[Thursday 30 September 2010] [15:30:51] <mikko> looking at this jzmq uses int64_t
[Thursday 30 September 2010] [15:31:27] <sustrik> gonzalo messed it up
[Thursday 30 September 2010] [15:31:55] <sustrik> please, do report the problem on jzmq bugtracker
[Thursday 30 September 2010] [15:33:53] <jasong_at_apache> docs want straight int
[Thursday 30 September 2010] [15:34:01] <mikko> done
[Thursday 30 September 2010] [15:34:07] <sustrik> thanks
[Thursday 30 September 2010] [15:35:28] * sustrik wonders how to make bindings work with both maint and stable without burdening maintainers with two separate branches
[Thursday 30 September 2010] [15:35:43] <sustrik> maint & master, i meant
[Thursday 30 September 2010] [15:35:53] <jasong_at_apache> maint? who needs maint!
[Thursday 30 September 2010] [15:36:00] <mikko> it’s a lot better now
[Thursday 30 September 2010] [15:36:01] <jasong_at_apache> ;-)
[Thursday 30 September 2010] [15:36:14] <mikko> as there aren’t breakages that often
[Thursday 30 September 2010] [15:36:40] <sustrik> yeah, the versioning contract helped
[Thursday 30 September 2010] [15:36:59] <mikko> are there constants for api version etc at the moment?
[Thursday 30 September 2010] [15:37:17] <sustrik> just a plain 0mq version
[Thursday 30 September 2010] [15:37:39] <sustrik> the api version should translate into library version
[Thursday 30 September 2010] [15:37:45] <sustrik> but that’s not done yet
[Thursday 30 September 2010] [15:37:51] <mikko> i guess it’s easy to create int out of that during configure phase
[Thursday 30 September 2010] [15:38:00] <mikko> something easy to compare in preprocessor macros
[Thursday 30 September 2010] [15:38:25] <mikko> something like #if ZMQ_VERSION_ID > 2001001 #else etc
[Thursday 30 September 2010] [15:38:31] <mikko> 2001001 being 2.1.1
[Thursday 30 September 2010] [15:38:33] <sustrik> mikko: good idea
[Thursday 30 September 2010] [15:41:06] <sustrik> on the other hand, some bindings (C#, lisp) don’t even include the header file
[Thursday 30 September 2010] [15:46:47] <mikko> hmm
[Thursday 30 September 2010] [15:47:02] <mikko> i reckon there still might be value for such a thing
[Thursday 30 September 2010] [15:47:12] <jasong_at_apache> everything compiled correctly (and I’m using trunk all around), also swapped int for int64 on the zmq_type
[Thursday 30 September 2010] [15:47:30] <jasong_at_apache> i meant the other way around there
[Thursday 30 September 2010] [15:49:09] <sustrik> goodo
[Thursday 30 September 2010] [15:49:36] <sustrik> anyway, this kind of thing is a bit scary as people are using the binding is production
[Thursday 30 September 2010] [15:49:40] <sustrik> in*
[Thursday 30 September 2010] [15:49:50] <jasong_at_apache> :-)
[Thursday 30 September 2010] [15:50:04] <sustrik> mikko: yes, it will help for most bindings
[Thursday 30 September 2010] [15:50:23] <sustrik> i’ll discuss it with mato tomorrow, he’s in charge of the build system
[Thursday 30 September 2010] [15:53:37] Quit omarkj has left this server (Quit: omarkj).
[Thursday 30 September 2010] [15:55:13] <novas0x2a> i’m using zmq_poll w/ 2.0.9, REQ/REP pair, tcp or ipc proto, same context (1 process). zmq_poll never seems to notice activity. if i use inproc it works, if i use a spinloop/nonblocking-recv (no poll) it works. should i expect it to work with tcp?
[Thursday 30 September 2010] [15:56:18] <sustrik> yes
[Thursday 30 September 2010] [15:56:46] <sustrik> if you have a probelm, write a simple program showing it and send it to the mailing list
[Thursday 30 September 2010] [15:57:29] <novas0x2a> could you do a quick sanity check to make sure i’m not pebkacing?
http://codepad.org/3Pxo5V1x[Thursday 30 September 2010] [15:58:45] <novas0x2a> (if you want to compile -DUSE_POLL or -DUSE_BLOCK)
[Thursday 30 September 2010] [16:00:31] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * rd215dac 10/ (src/Socket.cpp src/org/zeromq/ZMQ.java):
[Thursday 30 September 2010] [16:00:31] <CIA-20> jzmq: Added support for ZMQ_FD (getFD()) and ZMQ_EVENTS (getEvents()).
[Thursday 30 September 2010] [16:04:16] <sustrik> novas0x2a: the program looks ok after short inspection
[Thursday 30 September 2010] [16:04:57] <sustrik> if you want someone to actually debug it though, you have to strip as much code as possible and send it to the mailing list
[Thursday 30 September 2010] [16:10:51] <jond> novas0x2a: is this the error : raw.c:158: int main(int, char**): Assertion
`ret’ failed.?
[Thursday 30 September 2010] [16:11:39] Quit psino has left this server (Quit: psino).
[Thursday 30 September 2010] [16:11:52] <novas0x2a> jond: that’s the detection of the error, yeah (it should have printed Timeout just before that)
[Thursday 30 September 2010] [16:15:48] <jond> novas0x2a: yep but when you timeout in yr code, you return 0
[Thursday 30 September 2010] [16:15:50] <CIA-20> jzmq: 03Gonzalo Diethelm 07master * r9ccc16b 10/ src/org/zeromq/ZMQ.java : Added support for automatically growing the size of a Poller object. -
http://bit.ly/cCKiIK[Thursday 30 September 2010] [16:17:13] <jond> novas0x2a: and asserting on that doesnt seem to be a great idea
[Thursday 30 September 2010] [16:17:46] <novas0x2a> jond: this is test code, not real code. i’ve been trying to map the conditions under which it doesn’t work, to track it down. clearly, in real code, i would handle a timeout appropriately :)
[Thursday 30 September 2010] [16:19:09] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [16:20:22] Part cremes has left this channel.
[Thursday 30 September 2010] [16:20:33] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Thursday 30 September 2010] [16:20:33] Part cremes has left this channel.
[Thursday 30 September 2010] [16:21:15] <Peaceful> okay, so I made the hello world server-client example, and it worked great until I had to kill the server, and now the server won’t bind to tcp anymore. (???)
[Thursday 30 September 2010] [16:22:12] <Peaceful> ctrl-c wouldn’t stop the server, so I ended up looking up the pid and running “kill $pid”
[Thursday 30 September 2010] [16:22:39] <Peaceful> this has got to be some common newbie problem, I figure...
[Thursday 30 September 2010] [16:23:08] <Peaceful> But usually tcp ports become available again after you’ve killed the original program that was binding the port.
[Thursday 30 September 2010] [16:23:19] <Peaceful> ...in my experience, at least
[Thursday 30 September 2010] [16:25:01] <jasong_at_apache> is there a mailing list keeping track of commits? (does github do this?)
[Thursday 30 September 2010] [16:26:29] <mikko> github does that
[Thursday 30 September 2010] [16:28:02] <jond> novas0x2a: line 63 in yr file. you redefine rc. it should read rc = zmq_poll(item, 1, timeout); and the code works
[Thursday 30 September 2010] [16:30:16] <mikko> sustrik: java handles all options as int64_t (numeric ones)
[Thursday 30 September 2010] [16:30:28] <mikko> so i guess thats not an issue
[Thursday 30 September 2010] [16:31:00] <novas0x2a> jond: oh, man, i knew it was going to be my fault somehow. sigh.
[Thursday 30 September 2010] [16:31:13] <mikko> assuming that sizeof (uint64_t) <= sizeof (int64_t) is a sane assumption
[Thursday 30 September 2010] [16:32:48] <jasong_at_apache> is that only 64bit systems?
[Thursday 30 September 2010] [16:35:43] <mikko> sizeof both is 8 on systems i have access to
[Thursday 30 September 2010] [16:59:49] Quit stephank has left this server (Quit: urk IRC v0.-1.cvs -
http://urk.sf.net/).
[Thursday 30 September 2010] [17:17:37] Quit xla has left this server (Ping timeout: 255 seconds).
[Thursday 30 September 2010] [17:28:01] Quit drbobbeaty has left this server (Quit: drbobbeaty).
[Thursday 30 September 2010] [17:30:12] Quit rossij has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [17:32:02] Quit rgl has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [17:33:18] Join cremes has joined this channel (
~cremes@76.8.86.186).
[Thursday 30 September 2010] [17:57:40] Quit bgranger has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [18:07:45] Quit keffo has left this server (Ping timeout: 252 seconds).
[Thursday 30 September 2010] [18:09:54] Quit jond has left this server (Quit: Leaving).
[Thursday 30 September 2010] [18:14:20] Quit friendshi has left this server (Quit: friendshi).
[Thursday 30 September 2010] [18:15:55] Quit novas0x2a has left this server (Ping timeout: 240 seconds).
[Thursday 30 September 2010] [18:17:05] Quit keffo has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [18:18:55] Join novas0x2a has joined this channel (
~mike@womp.fihn.net).
[Thursday 30 September 2010] [18:22:13] Quit plq has left this server (Ping timeout: 245 seconds).
[Thursday 30 September 2010] [18:39:35] Join plq has joined this channel (
~plq@88.249.173.198).
[Thursday 30 September 2010] [19:10:34] * Peaceful doesn’t feel the love today. Oh well, I’m sure it will start working again after a reboot.
[Thursday 30 September 2010] [19:10:44] Quit Peaceful has left this server (Quit: Peaceful).
[Thursday 30 September 2010] [19:22:37] Quit clochette has left this server (Client Quit).
[Thursday 30 September 2010] [19:23:51] Part
js__ has left this channel.
[Thursday 30 September 2010] [19:53:48] Quit _rgl has left this server (Ping timeout: 272 seconds).
[Thursday 30 September 2010] [20:03:18] Quit friendshi has left this server (Remote host closed the connection).
[Thursday 30 September 2010] [20:37:15] Quit RobertLJ has left this server (Ping timeout: 265 seconds).
[Thursday 30 September 2010] [20:51:34] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [21:07:08] Quit xla has left this server (Ping timeout: 255 seconds).
[Thursday 30 September 2010] [21:42:04] Quit bbigras has left this server (Ping timeout: 252 seconds).
[Thursday 30 September 2010] [22:01:40] Quit ChicagoBud has left this server (Read error: Connection reset by peer).
[Thursday 30 September 2010] [22:18:26] Join ChicagoBud has joined this channel (
~bud@208.78.39.48).
[Thursday 30 September 2010] [22:18:54] Quit jasong_at_apache has left this server (Quit: jasong_at_apache).
[Thursday 30 September 2010] [23:05:45] Quit bgranger has left this server (Quit: bgranger).
[Thursday 30 September 2010] [23:10:48] Quit RobertLJ has left this server (Ping timeout: 240 seconds).
